-- Logs begin at Thu 2022-09-01 15:47:40 UTC, end at Thu 2022-09-01 17:01:50 UTC. -- Sep 01 16:16:37.675440 dv-cinder-852208-16-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 01 16:16:37.684472 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 01 16:16:37.686507 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 1 16:16:37 2022] *** Sep 01 16:16:37.686507 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 01 16:16:37.686507 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 01 16:16:37.686507 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: nodename: dv-cinder-852208-16-iscsi Sep 01 16:16:37.686507 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: machine: x86_64 Sep 01 16:16:37.686507 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: clock source: unix Sep 01 16:16:37.686507 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: pcre jit disabled Sep 01 16:16:37.686507 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: detected number of CPU cores: 4 Sep 01 16:16:37.686507 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: current working directory: / Sep 01 16:16:37.686507 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: detected binary path: /usr/bin/uwsgi-core Sep 01 16:16:37.687056 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: your processes number limit is 47680 Sep 01 16:16:37.687056 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: your memory page size is 4096 bytes Sep 01 16:16:37.687056 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: detected max file descriptor number: 2048 Sep 01 16:16:37.687056 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: lock engine: pthread robust mutexes Sep 01 16:16:37.687056 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: thunder lock: enabled Sep 01 16:16:37.687056 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 01 16:16:37.687056 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 01 16:16:37.725450 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: Python main interpreter initialized at 0x560ecac7a9c0 Sep 01 16:16:37.725450 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: python threads support enabled Sep 01 16:16:37.725450 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: your server socket listen backlog is limited to 100 connections Sep 01 16:16:37.725450 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: your mercy for graceful operations on workers is 90 seconds Sep 01 16:16:37.725813 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: mapped 403077 bytes (393 KB) for 2 cores Sep 01 16:16:37.725883 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: *** Operational MODE: preforking *** Sep 01 16:16:37.726133 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: *** uWSGI is running in multiple interpreter mode *** Sep 01 16:16:37.726133 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: spawned uWSGI master process (pid: 81490) Sep 01 16:16:37.726359 dv-cinder-852208-16-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 01 16:16:37.728842 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: spawned uWSGI worker 1 (pid: 81503, cores: 1) Sep 01 16:16:37.728842 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: spawned uWSGI worker 2 (pid: 81504, cores: 1) Sep 01 16:16:37.728842 dv-cinder-852208-16-iscsi devstack@keystone.service[81490]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 01 16:16:42.014049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 16:16:42.014049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: from cryptography.utils import int_from_bytes Sep 01 16:16:42.014868 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 16:16:42.014868 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: from cryptography.utils import int_from_bytes Sep 01 16:16:42.014998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 16:16:42.014998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: from cryptography.utils import int_from_bytes Sep 01 16:16:42.015406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 16:16:42.015406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: from cryptography.utils import int_from_bytes Sep 01 16:16:42.064363 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 16:16:42.064544 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 16:16:42.064840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] command line args: [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 16:16:42.065008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 16:16:42.065185 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ================================================================================ {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 16:16:42.065427 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 16:16:42.065523 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] admin_token = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.065608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 16:16:42.065690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] config_dir = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.065808 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] command line args: [] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 16:16:42.065895 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.066008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] config_source = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.066094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 16:16:42.066208 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] debug = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.066303 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ================================================================================ {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 16:16:42.066412 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.066672 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] admin_token = **** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.066739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] config_dir = [] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.066798 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.066928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.067013 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.067127 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] config_source = [] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.067214 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.067278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] debug = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.067409 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.067498 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.067636 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.067725 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.067829 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_config_append = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.067903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.067971 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.068096 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.068179 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_dir = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.068312 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_file = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.068392 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.068543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.068631 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] list_limit = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.068748 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] log_config_append = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.068867 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.068965 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.069049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.069125 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] log_dir = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.069233 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] log_file = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.069328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.069408 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.069482 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.069576 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.069690 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.069821 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.069923 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.070066 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.070163 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.070317 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.070399 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.070526 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.070650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.070764 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.070887 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.070982 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.071058 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.071756 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.071844 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.071993 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.072074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.072225 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.072318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.072455 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.072699 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.072789 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.58/identity {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.072905 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] publish_errors = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.072985 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.073135 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.073223 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.073363 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.073539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.073675 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.58/identity {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.073782 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.073922 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] publish_errors = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.074007 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.074133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.074261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] standard_threads = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.074368 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.074488 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.074574 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.074727 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.074806 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.074893 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.074968 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.075046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] use_journal = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.075180 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] standard_threads = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.075259 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] use_json = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.075384 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] use_stderr = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.075528 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.075605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] use_syslog = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.075678 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.075773 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.075853 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.075966 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] use_journal = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.076052 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.076135 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] use_json = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.076243 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.076330 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] use_stderr = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.076463 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] use_syslog = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.076542 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.076696 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:16:42.076777 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.076950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.077029 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.077182 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.077306 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.077425 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.077506 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.077657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.077744 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.077899 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.077979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.078133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.078253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.078401 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.078486 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.078632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.078721 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.078874 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.079020 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.079098 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.079171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.079333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.079420 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.079569 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.079648 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.079799 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.079884 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.080031 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.080113 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.080286 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.080367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.080528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.080608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.080762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.080847 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.080997 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.081081 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.081255 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.081335 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.081502 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.081591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.081746 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.081833 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.081974 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.082056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.082237 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.082337 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.082444 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.082526 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.082675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.082755 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.082918 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.083005 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.083160 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.083246 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.083386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.083466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.083611 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.083701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.083852 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.083939 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.084096 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.084176 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.084326 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.084416 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.084554 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.084641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.084778 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.084857 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.085009 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.085097 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.085237 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.085317 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.085463 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.085542 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.085687 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.085778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.085913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.085993 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.086139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.086238 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.086393 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.086551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.086628 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.086700 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.086899 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.087011 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.087634 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.087723 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.087856 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.087937 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.088087 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.088167 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.088317 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.088405 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.088558 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.088669 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.088789 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.088902 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.089023 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.089140 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.089240 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.089339 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.089478 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.089627 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.089722 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.089853 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.089931 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.089993 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.090135 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.090262 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.090396 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.090477 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.090624 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.090716 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.090856 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.090935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.091081 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.091162 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.091327 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.091411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.091558 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.091637 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.091786 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.091875 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.092012 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.092137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.092239 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.092329 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.092487 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.092567 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.092718 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.092803 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.092951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.093033 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.093185 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.093271 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.093404 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.093532 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.093632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.093714 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.093861 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.093942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.094108 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.094239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.094366 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.094489 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.094594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.094703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.094885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.095002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.095122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.095232 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.095352 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.095468 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.095594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.095704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.095834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.095953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.096077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.096192 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.096314 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.096437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.096558 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.096685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.096804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.096920 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.097044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.097044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.097250 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.097359 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.097470 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.097584 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.097716 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.097835 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.097957 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.098068 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.098186 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.098347 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.098581 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.098719 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.098843 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.098969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.099096 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.099238 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.099365 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.099453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.099529 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.099602 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.099675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.099747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.099825 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.099907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.099982 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.100111 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.100190 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.100263 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.100336 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.100417 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.100556 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.100644 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.100789 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.100872 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.101043 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.101131 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.101300 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.101382 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.101541 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.101621 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.101768 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.external = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.101937 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.102049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.102218 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.102311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.102475 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] auth.external = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.102588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.102794 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.102891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.password = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.103035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.token = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.103119 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.103273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.103356 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.103506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.103588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] auth.password = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.103751 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.103834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] auth.token = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.103981 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.104064 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.104216 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.104296 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.104463 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.104545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.104688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.104768 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.104921 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.105002 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.105163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.105246 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.105400 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.105485 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.105614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] credential.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.105694 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.105842 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.105925 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.106091 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.106181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.106344 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.106425 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.106587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.106671 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.106827 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.106907 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.107058 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.107146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.107300 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.107382 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.107554 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.107635 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.107790 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.107871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.108041 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.108124 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.108279 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.108359 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.108530 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.108611 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.108763 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.108846 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.108995 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.109077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.109237 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.109319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.109480 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.109563 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] federation.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.109709 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.109790 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.109952 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.110032 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.110183 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.110283 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.110444 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.110525 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.110676 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.110773 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.110917 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.110998 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.111148 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.111227 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.111390 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.111479 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.111621 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.111704 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.111865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.111946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.112110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.112266 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.112345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.112417 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.112562 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.112643 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.112804 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.112894 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.113034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.113121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.113266 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.113346 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.113509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.113596 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.113737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.113817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.113970 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.114050 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.114232 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.114331 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.114450 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.114533 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.114690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.114771 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.114917 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.115005 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.115147 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.115229 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.115391 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.115472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.115631 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.115720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.115874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.115955 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.116119 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.116201 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.116352 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.116442 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.116579 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.116661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.116811 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.116892 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.117036 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.117127 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.117267 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.117348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.117502 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.117583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.117729 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.117817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.117987 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.118081 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.118233 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.118318 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.118479 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.118573 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.118721 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.118802 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.118951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.119031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.119189 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.119277 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.119430 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.119512 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.119656 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.119737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.119887 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.119976 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.120110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.120192 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.120342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.120423 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.120574 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.120663 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.120801 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.120881 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.121030 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.121110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.121262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.121345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.121496 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.121576 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.121731 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.121812 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.121957 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.122039 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.122182 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.122281 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.122442 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.122524 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.122680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.122762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.122908 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.122989 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.123137 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.123217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.123369 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.123452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.123595 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.123675 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.123828 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.123907 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.124061 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.124144 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.124288 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.124370 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.124525 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.124610 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.124765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.124847 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.124997 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.125077 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.125225 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.125304 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.125475 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.125557 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.125720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.125801 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.125952 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.126046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.126180 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.126291 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.126442 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.126524 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.126674 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.126755 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.126907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.126997 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.127203 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.127285 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.127461 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.127543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.127707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.127818 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.127951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.128075 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.128196 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.128342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.128432 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.128576 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.128658 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.128809 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.128890 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.129040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.129129 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.129270 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.129351 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.129548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.129628 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.129780 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.129890 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.130021 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.130128 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.130289 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.130404 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.130527 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.130683 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.130766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.130925 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.131007 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.131156 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.131243 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.131386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.131469 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.131618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.131699 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.131889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.131969 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.132132 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.132215 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.132371 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.132456 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.132600 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.132681 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.132833 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.132915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.133071 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] resource.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.133152 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.133318 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.133399 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.133563 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.134140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.134346 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.134430 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.134594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.134674 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.134826 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.134897 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.135066 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.135145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.135299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.135379 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.135530 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.135606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.135756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] role.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.135837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.135987 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] role.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.136075 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] role.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.136216 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.136296 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] role.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.136449 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.136529 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.136693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.136782 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.136936 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.137017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.137179 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.137259 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.137422 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.137511 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.137664 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.137744 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.137906 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.137985 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.138148 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.138262 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.138403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.138486 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.138638 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.138718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.138882 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.138970 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.139124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.139205 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.139366 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.139445 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.139609 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.139757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.139989 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.140068 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.140224 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.140303 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.140468 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.140556 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.140714 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.140796 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.140948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.141029 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.141174 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.141265 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.141416 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.141497 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.141660 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.141740 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.141894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.141977 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.142122 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.142264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.142370 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.142453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.142597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.142682 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.142842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.142924 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.143092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.143174 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.143337 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.143420 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.143569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.143650 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.143801 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.143883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.144024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.144108 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.144261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.144343 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.144492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.144573 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] token.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.144726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.144808 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.144954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.145034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.145200 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.145328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.145438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.145521 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.145672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.145754 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.145906 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.145988 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.146143 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.146243 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.146407 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.146488 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.146639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.146730 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.146873 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.146956 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.147102 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.147181 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.147334 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.147420 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.147577 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.147660 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.147817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.147906 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.148055 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.148139 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.148301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.148391 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.148556 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.connection = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.148645 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.148794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.148875 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.connection = **** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.149034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.149116 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.149347 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.149947 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.150125 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.150296 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.150366 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.150427 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.150600 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.150700 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.150832 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.150914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.151063 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.151144 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.151301 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.151391 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.151562 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.151654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.151779 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.151859 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.151986 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.152077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.152228 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.152310 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.152457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.152538 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.152696 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.152786 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.152921 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.153003 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.153150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.153231 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.153376 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.153465 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.153543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 16:16:42.153617 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:16:42.153763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 16:16:42.182740 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.182970 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.183056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.183194 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.183281 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.183446 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.191367 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.191991 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.219870 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.220098 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.220377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.220608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.273065 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.273247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.321004 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.321841 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.322707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.322942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.475947 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.476178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.476383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.476476 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.476601 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.476689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.476816 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.476902 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.477029 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.477125 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.477247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.478082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.500729 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.500840 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.500953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.501080 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.501167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.501309 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.501395 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.501533 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.501628 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.501754 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.501840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.501970 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.502056 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.502187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.512905 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.513445 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.526153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.526342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.526566 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.526667 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.526782 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.526869 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.526993 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.527079 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.527203 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.527290 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.527413 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.527498 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.527686 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.527903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:16:42.544769 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.544980 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 16:16:42.545324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.545408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.545522 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 16:16:42.545622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 16:16:42.545789 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.545883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.546048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.546134 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 16:16:42.546332 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.546421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.546593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.546855 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.547123 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.547390 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.556116 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.556395 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.556670 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.556753 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.556951 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.557051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.557260 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.557542 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.563882 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.564486 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.566764 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.567046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.567328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.567412 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.567609 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.567694 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.567891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.567978 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.568153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.568237 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.568420 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.568513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.568689 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.568775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.568959 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.569041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.569226 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.569347 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.569490 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.569628 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.569907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.570180 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.592405 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.593075 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.593860 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.594143 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.594561 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.594849 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.598736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.599021 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.599302 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.599497 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.599785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.600065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.606370 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.606655 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.606993 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.607282 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.610666 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.610959 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.611251 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.611400 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.611539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.611686 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.611817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.611962 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.612257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.612535 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.612821 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.613091 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.613374 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.613744 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.614034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.614359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.629773 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.630049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.630464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.630745 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.634458 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.635127 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.637529 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.637824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.638357 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.638656 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.642142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.642833 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.645258 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.645939 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.648343 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.648641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.648926 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.649164 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.649250 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.650020 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.650180 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.650331 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.650459 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.650627 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.650897 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.651175 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.666328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.666614 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.666886 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.667160 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.667245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.667522 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.667793 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.668057 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.673827 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.674899 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.675500 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.676417 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.677021 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.677298 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.677957 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.678256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.679951 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.680236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.680541 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.680826 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.680919 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.681130 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.681224 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.681561 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.681859 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.682158 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.692280 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.692565 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.692846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.693126 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.693409 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.693514 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.693692 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.693787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.693974 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.694173 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.694481 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.694766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.695046 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.695336 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.709680 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.711155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.712750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.714102 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.715650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.717490 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.717635 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.718954 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.719078 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.719368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.719650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.720564 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.720852 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.721145 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.726126 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.727484 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.727645 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.728962 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.730533 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.730827 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.731114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.731393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.731970 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.732261 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.732544 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.732822 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.739273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.739563 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.739845 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.740597 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.740901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.741161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.746705 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.747000 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.747284 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.747558 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.747829 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.748071 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:16:42.748168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.748355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.748440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.748633 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.748715 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.748905 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.748996 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.749176 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.749269 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.749444 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.749537 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.749704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.749788 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.749966 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.750050 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.750259 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.750359 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.750547 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.750820 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.751080 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.751345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.751613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:16:42.784758 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x560ecac7a9c0 pid: 81503 (default app) Sep 01 16:16:42.786579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x560ecac7a9c0 pid: 81504 (default app) Sep 01 16:16:42.788204 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bfcfa-1989-4af8-a5ec-bcbcb6410876 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:42.788491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bfcfa-1989-4af8-a5ec-bcbcb6410876 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:42.788766 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bfcfa-1989-4af8-a5ec-bcbcb6410876 None None] PATH_INFO: `/v3` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:42.793657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1/1] 10.222.0.58 () {50 vars in 791 bytes} [Thu Sep 1 16:16:42 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:49.392567 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376896f-ccf1-4d92-bce0-2afe02812705 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:49.393137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376896f-ccf1-4d92-bce0-2afe02812705 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:49.393137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376896f-ccf1-4d92-bce0-2afe02812705 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:49.397259 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1/2] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:16:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:16:49.402974 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e243113-9d2e-48dc-8e06-67c6c935664a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:49.403451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e243113-9d2e-48dc-8e06-67c6c935664a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:49.403594 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e243113-9d2e-48dc-8e06-67c6c935664a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:49.479377 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: INFO dbcounter [None req-2e243113-9d2e-48dc-8e06-67c6c935664a None None] Registered counter for database keystone Sep 01 16:16:49.499174 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG oslo_db.sqlalchemy.engines [None req-2e243113-9d2e-48dc-8e06-67c6c935664a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81503) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 16:16:49.505426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writer thread running {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 16:16:49.682543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG passlib.handlers.bcrypt [None req-2e243113-9d2e-48dc-8e06-67c6c935664a None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81503) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 16:16:49.682953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG passlib.handlers.bcrypt [None req-2e243113-9d2e-48dc-8e06-67c6c935664a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81503) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 16:16:49.740351 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2e243113-9d2e-48dc-8e06-67c6c935664a None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:16:49.741934 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2e243113-9d2e-48dc-8e06-67c6c935664a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:16:49.882185 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 2/3] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:16:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 480 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 01 16:16:50.351628 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6376896f-ccf1-4d92-bce0-2afe02812705 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:50.421321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: INFO dbcounter [None req-6376896f-ccf1-4d92-bce0-2afe02812705 None None] Registered counter for database keystone Sep 01 16:16:50.443027 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG oslo_db.sqlalchemy.engines [None req-6376896f-ccf1-4d92-bce0-2afe02812705 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81504) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 16:16:50.449341 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writer thread running {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 16:16:50.575240 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6376896f-ccf1-4d92-bce0-2afe02812705 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:50.580374 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae88735-ee79-4924-bdeb-283f9a2beba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:50.581012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae88735-ee79-4924-bdeb-283f9a2beba0 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:50.581290 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae88735-ee79-4924-bdeb-283f9a2beba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:50.581562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae88735-ee79-4924-bdeb-283f9a2beba0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:50.589748 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae88735-ee79-4924-bdeb-283f9a2beba0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:50.642314 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fae88735-ee79-4924-bdeb-283f9a2beba0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 16:16:50.642588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae88735-ee79-4924-bdeb-283f9a2beba0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:50.652374 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 2/4] 10.222.0.58 () {58 vars in 1324 bytes} [Thu Sep 1 16:16:50 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 16:16:50.658554 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e243113-9d2e-48dc-8e06-67c6c935664a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:50.673606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e243113-9d2e-48dc-8e06-67c6c935664a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:50.677998 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b9593-2cdc-4c8a-a5bf-401edb07c515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:50.679141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b9593-2cdc-4c8a-a5bf-401edb07c515 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:50.679427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b9593-2cdc-4c8a-a5bf-401edb07c515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:50.679703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b9593-2cdc-4c8a-a5bf-401edb07c515 admin admin] PATH_INFO: `/v3/projects/e2c38221adde4053b2b37201b4051903` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:50.681869 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5b9593-2cdc-4c8a-a5bf-401edb07c515 admin admin] RBAC: Authorizing `identity:get_project(project_id=e2c38221adde4053b2b37201b4051903)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:50.734287 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ce5b9593-2cdc-4c8a-a5bf-401edb07c515 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 16:16:50.734640 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5b9593-2cdc-4c8a-a5bf-401edb07c515 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:50.739362 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 3/5] 10.222.0.58 () {56 vars in 1207 bytes} [Thu Sep 1 16:16:50 2022] GET /identity/v3/projects/e2c38221adde4053b2b37201b4051903 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:52.624552 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b3127-9fa1-4a9a-a0b9-f57a5c4f2c9d None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:52.625088 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b3127-9fa1-4a9a-a0b9-f57a5c4f2c9d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:52.625088 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b3127-9fa1-4a9a-a0b9-f57a5c4f2c9d None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:52.629208 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 3/6] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:16:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:16:52.634611 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab78543-8a8c-419b-987e-af902e30fd20 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:52.635007 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab78543-8a8c-419b-987e-af902e30fd20 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:52.635170 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab78543-8a8c-419b-987e-af902e30fd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:52.655322 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5ab78543-8a8c-419b-987e-af902e30fd20 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:16:52.656818 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5ab78543-8a8c-419b-987e-af902e30fd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:16:52.664704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 4/7] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:16:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 01 16:16:52.939422 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b3127-9fa1-4a9a-a0b9-f57a5c4f2c9d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:52.951860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b3127-9fa1-4a9a-a0b9-f57a5c4f2c9d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:52.955928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35daed9-4d6e-4bc5-b8f3-04588f4ae895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:52.956516 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35daed9-4d6e-4bc5-b8f3-04588f4ae895 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:52.956794 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35daed9-4d6e-4bc5-b8f3-04588f4ae895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:52.957066 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35daed9-4d6e-4bc5-b8f3-04588f4ae895 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:52.965229 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35daed9-4d6e-4bc5-b8f3-04588f4ae895 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:52.967029 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35daed9-4d6e-4bc5-b8f3-04588f4ae895 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:52.975475 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 4/8] 10.222.0.58 () {58 vars in 1324 bytes} [Thu Sep 1 16:16:52 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:16:52.980654 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab78543-8a8c-419b-987e-af902e30fd20 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:52.989582 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab78543-8a8c-419b-987e-af902e30fd20 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:52.993922 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b04f4e-7936-4f33-b549-5fbebcaf7f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:52.994575 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b04f4e-7936-4f33-b549-5fbebcaf7f52 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:52.994857 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b04f4e-7936-4f33-b549-5fbebcaf7f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:52.995131 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b04f4e-7936-4f33-b549-5fbebcaf7f52 admin admin] PATH_INFO: `/v3/users/9fe3a5506b35415a9373071c0b976e34` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:52.996440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b04f4e-7936-4f33-b549-5fbebcaf7f52 admin admin] RBAC: Authorizing `identity:get_user(user_id=9fe3a5506b35415a9373071c0b976e34)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:52.998270 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b04f4e-7936-4f33-b549-5fbebcaf7f52 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:52.999771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 5/9] 10.222.0.58 () {56 vars in 1198 bytes} [Thu Sep 1 16:16:52 2022] GET /identity/v3/users/9fe3a5506b35415a9373071c0b976e34 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:54.971764 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e2160-7e20-4f6c-8d2c-67fc3f54c824 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:54.972313 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e2160-7e20-4f6c-8d2c-67fc3f54c824 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:54.972313 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e2160-7e20-4f6c-8d2c-67fc3f54c824 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:54.973644 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 5/10] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:16:54.978713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8f961-9001-473d-be7f-d49cd39cc047 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:54.979110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8f961-9001-473d-be7f-d49cd39cc047 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:54.979263 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8f961-9001-473d-be7f-d49cd39cc047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:54.997519 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0fe8f961-9001-473d-be7f-d49cd39cc047 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:16:54.998722 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0fe8f961-9001-473d-be7f-d49cd39cc047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:16:55.010538 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d3091-ce21-405b-bedc-c653e5bd0bce None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:55.010828 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d3091-ce21-405b-bedc-c653e5bd0bce None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:55.011107 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d3091-ce21-405b-bedc-c653e5bd0bce None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:55.012421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 6/11] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:16:55.017332 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b42387-c5e2-4495-a964-0a89069bb8f0 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:55.017620 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b42387-c5e2-4495-a964-0a89069bb8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:55.017900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b42387-c5e2-4495-a964-0a89069bb8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:55.063161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG passlib.handlers.bcrypt [None req-93b42387-c5e2-4495-a964-0a89069bb8f0 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81504) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 16:16:55.063572 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG passlib.handlers.bcrypt [None req-93b42387-c5e2-4495-a964-0a89069bb8f0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81504) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 16:16:55.079513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 6/12] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:16:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:16:55.112518 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-93b42387-c5e2-4495-a964-0a89069bb8f0 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:16:55.114108 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-93b42387-c5e2-4495-a964-0a89069bb8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:16:55.213564 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 7/13] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:16:55.347200 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8f961-9001-473d-be7f-d49cd39cc047 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:55.371879 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:55.451260 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8f961-9001-473d-be7f-d49cd39cc047 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:55.474395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b42387-c5e2-4495-a964-0a89069bb8f0 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:55.485033 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db0b5d-5967-4d70-8c2f-5cfe03744081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:55.485693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db0b5d-5967-4d70-8c2f-5cfe03744081 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:55.485975 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db0b5d-5967-4d70-8c2f-5cfe03744081 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:55.486277 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db0b5d-5967-4d70-8c2f-5cfe03744081 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:55.497093 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db0b5d-5967-4d70-8c2f-5cfe03744081 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:16:55.511160 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-c5db0b5d-5967-4d70-8c2f-5cfe03744081 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:16:55.511160 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:16:55.511160 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:16:55.511160 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:16:55.511160 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:16:55.511160 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:16:55.511160 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:16:55.511160 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:16:55.511160 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:16:55.511160 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:16:55.511160 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:16:55.511160 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:16:55.511160 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:16:55.511160 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:16:55.511160 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:16:55.511160 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:16:55.512321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:16:55.512321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:16:55.512321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:16:55.512321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:16:55.512321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:16:55.512321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:16:55.512321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:16:55.512321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:16:55.512321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:16:55.512321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:16:55.512321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:16:55.512321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:16:55.512321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:16:55.512321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:16:55.512321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:16:55.512321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:16:55.513539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:16:55.513539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:16:55.513539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:16:55.513539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:16:55.513539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:16:55.513539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:16:55.513539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:16:55.513539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:16:55.513539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:16:55.513539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:16:55.513539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:16:55.513539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:16:55.513539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 7/14] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:16:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:16:55.518013 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db0b5d-5967-4d70-8c2f-5cfe03744081 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:55.580679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b42387-c5e2-4495-a964-0a89069bb8f0 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:55.583047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:55.602264 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:55.619291 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8c30a-cbc4-44db-8af8-5eeb3ff69c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:55.620152 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8c30a-cbc4-44db-8af8-5eeb3ff69c2d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:55.620438 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8c30a-cbc4-44db-8af8-5eeb3ff69c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:55.620714 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8c30a-cbc4-44db-8af8-5eeb3ff69c2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:55.620908 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db0b5d-5967-4d70-8c2f-5cfe03744081 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:55.655037 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3d40d-66cd-4931-b418-ad2ef8619b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:55.655642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3d40d-66cd-4931-b418-ad2ef8619b3c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:55.655924 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3d40d-66cd-4931-b418-ad2ef8619b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:55.656200 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3d40d-66cd-4931-b418-ad2ef8619b3c None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:55.656872 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3d40d-66cd-4931-b418-ad2ef8619b3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:16:55.657180 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3d40d-66cd-4931-b418-ad2ef8619b3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:55.658607 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3d40d-66cd-4931-b418-ad2ef8619b3c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:55.670975 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 8/15] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:16:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:16:55.676694 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3d40d-66cd-4931-b418-ad2ef8619b3c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:55.689121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8c30a-cbc4-44db-8af8-5eeb3ff69c2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:55.690817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8c30a-cbc4-44db-8af8-5eeb3ff69c2d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:55.773613 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3d40d-66cd-4931-b418-ad2ef8619b3c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:55.793218 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 8/16] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:16:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 16:16:55.798777 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8c30a-cbc4-44db-8af8-5eeb3ff69c2d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:55.801537 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:55.808509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb2212-eab4-40de-834f-f8ba49bbe172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:55.809148 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb2212-eab4-40de-834f-f8ba49bbe172 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:55.809438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb2212-eab4-40de-834f-f8ba49bbe172 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:55.809720 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb2212-eab4-40de-834f-f8ba49bbe172 None admin] PATH_INFO: `/v3/users/9fe3a5506b35415a9373071c0b976e34` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:55.810886 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecb2212-eab4-40de-834f-f8ba49bbe172 None admin] RBAC: Authorizing `identity:get_user(user_id=9fe3a5506b35415a9373071c0b976e34)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:55.812348 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecb2212-eab4-40de-834f-f8ba49bbe172 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:55.813920 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 9/17] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:16:55 2022] GET /identity/v3/users/9fe3a5506b35415a9373071c0b976e34 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:16:55.819443 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb2212-eab4-40de-834f-f8ba49bbe172 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:55.833724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:55.898912 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8c30a-cbc4-44db-8af8-5eeb3ff69c2d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:55.919819 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb2212-eab4-40de-834f-f8ba49bbe172 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:55.932679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174f836-cd17-4661-8aeb-e24df8cdb311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:55.933531 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174f836-cd17-4661-8aeb-e24df8cdb311 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:55.933813 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174f836-cd17-4661-8aeb-e24df8cdb311 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:55.934085 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174f836-cd17-4661-8aeb-e24df8cdb311 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:55.953648 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8174f836-cd17-4661-8aeb-e24df8cdb311 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:55.954542 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f0e899-c846-43ac-ae08-0070e1d6023c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:55.955170 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f0e899-c846-43ac-ae08-0070e1d6023c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:55.955263 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8174f836-cd17-4661-8aeb-e24df8cdb311 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:55.955454 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f0e899-c846-43ac-ae08-0070e1d6023c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:55.955777 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f0e899-c846-43ac-ae08-0070e1d6023c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:55.956841 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f0e899-c846-43ac-ae08-0070e1d6023c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:55.958313 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f0e899-c846-43ac-ae08-0070e1d6023c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:55.960005 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 10/18] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:16:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:16:55.965701 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f0e899-c846-43ac-ae08-0070e1d6023c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:55.972817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-8174f836-cd17-4661-8aeb-e24df8cdb311 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:16:55.972817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:16:55.972817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:16:55.972817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:16:55.972817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:16:55.972817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:16:55.972817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:16:55.972817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:16:55.972817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:16:55.972817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:16:55.972817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:16:55.972817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:16:55.972817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:16:55.972817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:16:55.972817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:16:55.972817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:16:55.974110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:16:55.974110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:16:55.974110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:16:55.974110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:16:55.974110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:16:55.974110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:16:55.974110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:16:55.974110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:16:55.974110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:16:55.974110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:16:55.974110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:16:55.974110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:16:55.974110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:16:55.974110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:16:55.974110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:16:55.974110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:16:55.974110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:16:55.974997 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:16:55.974997 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:16:55.974997 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:16:55.974997 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 9/19] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:16:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:16:55.979981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174f836-cd17-4661-8aeb-e24df8cdb311 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:56.036805 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:56.049647 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:56.072465 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f0e899-c846-43ac-ae08-0070e1d6023c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:56.085027 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174f836-cd17-4661-8aeb-e24df8cdb311 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:56.106531 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f4b21-6a77-4892-bedd-4287e29f4540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:56.107138 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f4b21-6a77-4892-bedd-4287e29f4540 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:56.107419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f4b21-6a77-4892-bedd-4287e29f4540 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:56.107696 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f4b21-6a77-4892-bedd-4287e29f4540 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:56.108421 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212f4b21-6a77-4892-bedd-4287e29f4540 None admin] RBAC: Adding query filter params (role.id=ff293acb8551463aafe6970230042b89, scope.domain.id=default, user.id=9fe3a5506b35415a9373071c0b976e34) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:16:56.108728 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212f4b21-6a77-4892-bedd-4287e29f4540 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:56.110164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212f4b21-6a77-4892-bedd-4287e29f4540 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:56.116625 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 11/20] 10.222.0.58 () {56 vars in 1322 bytes} [Thu Sep 1 16:16:55 2022] GET /identity/v3/role_assignments?user.id=9fe3a5506b35415a9373071c0b976e34&scope.domain.id=default&role.id=ff293acb8551463aafe6970230042b89 => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:16:56.120193 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b557a-44ad-4580-b2b3-f832e66592de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:56.120883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b557a-44ad-4580-b2b3-f832e66592de None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:56.121228 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b557a-44ad-4580-b2b3-f832e66592de None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:56.121567 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b557a-44ad-4580-b2b3-f832e66592de None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:56.122340 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25b557a-44ad-4580-b2b3-f832e66592de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:16:56.122712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25b557a-44ad-4580-b2b3-f832e66592de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:56.124139 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25b557a-44ad-4580-b2b3-f832e66592de None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:56.144797 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 10/21] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:16:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:16:58.167055 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39720695-8c86-4008-a20a-4bc7114159d8 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:58.167572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39720695-8c86-4008-a20a-4bc7114159d8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:58.167656 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39720695-8c86-4008-a20a-4bc7114159d8 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:58.169273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 12/22] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:16:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:16:58.174366 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4bf3f-469b-4d62-bfd8-1342b1e80a3f None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:58.175202 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4bf3f-469b-4d62-bfd8-1342b1e80a3f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:58.175691 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4bf3f-469b-4d62-bfd8-1342b1e80a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:58.206051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-edf4bf3f-469b-4d62-bfd8-1342b1e80a3f None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:16:58.207596 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-edf4bf3f-469b-4d62-bfd8-1342b1e80a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:16:58.292616 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 11/23] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:16:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:16:58.591504 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38750d3f-dd5e-4dc1-bb3c-3c1c308a506d None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:58.592288 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38750d3f-dd5e-4dc1-bb3c-3c1c308a506d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:58.592772 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38750d3f-dd5e-4dc1-bb3c-3c1c308a506d None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:58.594323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 13/24] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:16:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:16:58.597839 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a783b2-06f5-4b2c-98b6-dc9d84f87b77 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:58.598122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a783b2-06f5-4b2c-98b6-dc9d84f87b77 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:58.598499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a783b2-06f5-4b2c-98b6-dc9d84f87b77 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:58.599812 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 12/25] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:16:58.600399 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fc79c-b2d5-4641-971a-b264e6730741 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:58.601140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fc79c-b2d5-4641-971a-b264e6730741 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:58.601623 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fc79c-b2d5-4641-971a-b264e6730741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:58.606542 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6b24c-10e7-43e0-af24-93fcec3bccb9 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:58.606826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6b24c-10e7-43e0-af24-93fcec3bccb9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:58.607102 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6b24c-10e7-43e0-af24-93fcec3bccb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:58.624214 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1a0fc79c-b2d5-4641-971a-b264e6730741 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:16:58.625782 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1a0fc79c-b2d5-4641-971a-b264e6730741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:16:58.641964 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-0fe6b24c-10e7-43e0-af24-93fcec3bccb9 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:16:58.643424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0fe6b24c-10e7-43e0-af24-93fcec3bccb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:16:58.738816 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 14/26] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:16:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:16:58.739888 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0fc79c-b2d5-4641-971a-b264e6730741 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:58.742612 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:58.802115 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 13/27] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:16:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 01 16:16:58.868333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:58.904306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0fc79c-b2d5-4641-971a-b264e6730741 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:58.956749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af169cb-a585-489c-bd83-76e490856536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:58.957390 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af169cb-a585-489c-bd83-76e490856536 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:58.957672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af169cb-a585-489c-bd83-76e490856536 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:58.957947 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af169cb-a585-489c-bd83-76e490856536 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:58.974266 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af169cb-a585-489c-bd83-76e490856536 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:16:58.979685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-5af169cb-a585-489c-bd83-76e490856536 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:16:58.979685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:16:58.979685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:16:58.979685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:16:58.979685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:16:58.979685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:16:58.979685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:16:58.979685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:16:58.979685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:16:58.979685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:16:58.979685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:16:58.979685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:16:58.979685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:16:58.979685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:16:58.979685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:16:58.979685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:16:58.980538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:16:58.980538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:16:58.980538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:16:58.980538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:16:58.980538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:16:58.980538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:16:58.980538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:16:58.980538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:16:58.980538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:16:58.980538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:16:58.980538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:16:58.980538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:16:58.980538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:16:58.980538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:16:58.980538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:16:58.980538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:16:58.981319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:16:58.981319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:16:58.981319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:16:58.981319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:16:58.981319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:16:58.981319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:16:58.981319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:16:58.981319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:16:58.981319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:16:58.981319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:16:58.981319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:16:58.981319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:16:58.985333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 15/28] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:16:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:16:58.986608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe6b24c-10e7-43e0-af24-93fcec3bccb9 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:59.127146 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af169cb-a585-489c-bd83-76e490856536 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:59.174773 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe6b24c-10e7-43e0-af24-93fcec3bccb9 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:59.241115 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d477dc7-98c0-4132-90b1-0132199d90ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:59.241741 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d477dc7-98c0-4132-90b1-0132199d90ad None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:59.242292 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d477dc7-98c0-4132-90b1-0132199d90ad None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:59.242577 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d477dc7-98c0-4132-90b1-0132199d90ad None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:59.244241 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d477dc7-98c0-4132-90b1-0132199d90ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:16:59.244816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d477dc7-98c0-4132-90b1-0132199d90ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:59.246221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d477dc7-98c0-4132-90b1-0132199d90ad None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:59.264197 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 14/29] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:16:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:59.265729 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d477dc7-98c0-4132-90b1-0132199d90ad None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:59.290949 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af169cb-a585-489c-bd83-76e490856536 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:59.326266 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:59.340033 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c5c890-83dd-4ab1-a2fa-83fd24aa4cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:59.340672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5c890-83dd-4ab1-a2fa-83fd24aa4cae None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:59.340958 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5c890-83dd-4ab1-a2fa-83fd24aa4cae None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:59.341231 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5c890-83dd-4ab1-a2fa-83fd24aa4cae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:59.344195 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:59.356693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c5c890-83dd-4ab1-a2fa-83fd24aa4cae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:59.358395 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c5c890-83dd-4ab1-a2fa-83fd24aa4cae None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:59.375124 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-62c5c890-83dd-4ab1-a2fa-83fd24aa4cae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:16:59.375124 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:16:59.375124 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:16:59.375124 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:16:59.375124 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:16:59.375124 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:16:59.375124 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:16:59.375124 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:16:59.375124 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:16:59.375124 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:16:59.375124 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:16:59.375124 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:16:59.375124 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:16:59.375124 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:16:59.375124 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:16:59.375124 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:16:59.376025 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:16:59.376025 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:16:59.376025 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:16:59.376025 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:16:59.376025 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:16:59.376025 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:16:59.376025 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:16:59.376025 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:16:59.376025 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:16:59.376025 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:16:59.376025 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:16:59.376025 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:16:59.376025 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:16:59.376025 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:16:59.376025 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:16:59.376025 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:16:59.376025 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:16:59.376862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:16:59.376862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:16:59.376862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:16:59.378708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 16/30] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:16:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:16:59.380263 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71821bf1-92fa-46ba-9fe2-bd186c196a32 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:59.380742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71821bf1-92fa-46ba-9fe2-bd186c196a32 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:59.381209 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71821bf1-92fa-46ba-9fe2-bd186c196a32 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:59.382717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 17/31] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:16:59.383736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71821bf1-92fa-46ba-9fe2-bd186c196a32 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:59.414299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d477dc7-98c0-4132-90b1-0132199d90ad None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:59.461675 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767ec5b-3c65-4e80-a63f-54074bd63883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:59.462895 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767ec5b-3c65-4e80-a63f-54074bd63883 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:59.463184 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767ec5b-3c65-4e80-a63f-54074bd63883 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:59.463473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767ec5b-3c65-4e80-a63f-54074bd63883 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:59.464134 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c767ec5b-3c65-4e80-a63f-54074bd63883 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:59.465785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c767ec5b-3c65-4e80-a63f-54074bd63883 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:59.507677 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71821bf1-92fa-46ba-9fe2-bd186c196a32 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:59.556635 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb03b5b-e649-4587-98f7-8e8b1f46aa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:59.557395 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb03b5b-e649-4587-98f7-8e8b1f46aa99 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:59.557686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb03b5b-e649-4587-98f7-8e8b1f46aa99 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:59.557968 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb03b5b-e649-4587-98f7-8e8b1f46aa99 None admin] PATH_INFO: `/v3/users/9fe3a5506b35415a9373071c0b976e34` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:59.559186 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb03b5b-e649-4587-98f7-8e8b1f46aa99 None admin] RBAC: Authorizing `identity:get_user(user_id=9fe3a5506b35415a9373071c0b976e34)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:59.560654 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb03b5b-e649-4587-98f7-8e8b1f46aa99 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:59.562444 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 18/32] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:16:59 2022] GET /identity/v3/users/9fe3a5506b35415a9373071c0b976e34 => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:59.563221 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb03b5b-e649-4587-98f7-8e8b1f46aa99 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:59.610357 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 15/33] 10.222.0.58 () {60 vars in 1131 bytes} [Thu Sep 1 16:16:59 2022] POST /identity/v3/roles => generated 219 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:16:59.612294 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaff4a6-e9fe-4a98-9498-1b3e6fff3de7 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:59.612814 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaff4a6-e9fe-4a98-9498-1b3e6fff3de7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:59.613305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaff4a6-e9fe-4a98-9498-1b3e6fff3de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:59.632858 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:59.633537 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2eaff4a6-e9fe-4a98-9498-1b3e6fff3de7 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:16:59.635853 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2eaff4a6-e9fe-4a98-9498-1b3e6fff3de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:16:59.727511 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb03b5b-e649-4587-98f7-8e8b1f46aa99 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:59.783100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d757fab-9337-4861-80a4-809bb05aa132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:59.784047 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d757fab-9337-4861-80a4-809bb05aa132 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:59.784527 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d757fab-9337-4861-80a4-809bb05aa132 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:59.785013 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d757fab-9337-4861-80a4-809bb05aa132 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:59.785901 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d757fab-9337-4861-80a4-809bb05aa132 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:16:59.786168 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 16/34] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:16:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:16:59.786713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d757fab-9337-4861-80a4-809bb05aa132 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:59.787056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaff4a6-e9fe-4a98-9498-1b3e6fff3de7 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:59.788682 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d757fab-9337-4861-80a4-809bb05aa132 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:59.810984 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 19/35] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:59.812816 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f785fc-2ca2-4fe0-bd91-8e775c01e03e None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:59.813456 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f785fc-2ca2-4fe0-bd91-8e775c01e03e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:59.813967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f785fc-2ca2-4fe0-bd91-8e775c01e03e None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:59.815437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 20/36] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:16:59.816845 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f785fc-2ca2-4fe0-bd91-8e775c01e03e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:59.825363 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:59.917745 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaff4a6-e9fe-4a98-9498-1b3e6fff3de7 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:59.943644 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f785fc-2ca2-4fe0-bd91-8e775c01e03e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:59.957210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee6209-99f4-4b7a-a9f6-48673e35230b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:59.958140 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee6209-99f4-4b7a-a9f6-48673e35230b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:59.958666 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee6209-99f4-4b7a-a9f6-48673e35230b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:59.959191 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee6209-99f4-4b7a-a9f6-48673e35230b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:59.959962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:59.962442 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee6209-99f4-4b7a-a9f6-48673e35230b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:59.965286 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee6209-99f4-4b7a-a9f6-48673e35230b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:59.967324 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 17/37] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:16:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:59.968836 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da3e15-efcf-49d5-bca0-1aae9bf06c66 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:59.969356 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da3e15-efcf-49d5-bca0-1aae9bf06c66 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:59.969820 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da3e15-efcf-49d5-bca0-1aae9bf06c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:59.982696 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66abb1-e677-4e0d-aa9a-03ceec5e9257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:59.983317 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66abb1-e677-4e0d-aa9a-03ceec5e9257 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:59.983657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66abb1-e677-4e0d-aa9a-03ceec5e9257 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:59.983937 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66abb1-e677-4e0d-aa9a-03ceec5e9257 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:59.984600 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac66abb1-e677-4e0d-aa9a-03ceec5e9257 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:59.986323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac66abb1-e677-4e0d-aa9a-03ceec5e9257 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:59.990895 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e8da3e15-efcf-49d5-bca0-1aae9bf06c66 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:16:59.992424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e8da3e15-efcf-49d5-bca0-1aae9bf06c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:00.084381 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 18/38] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:16:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:00.091276 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da3e15-efcf-49d5-bca0-1aae9bf06c66 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:00.124392 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:00.137008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 21/39] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Sep 1 16:16:59 2022] POST /identity/v3/projects => generated 301 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:17:00.138155 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66abb1-e677-4e0d-aa9a-03ceec5e9257 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:00.219892 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da3e15-efcf-49d5-bca0-1aae9bf06c66 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:00.254697 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66abb1-e677-4e0d-aa9a-03ceec5e9257 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:00.261537 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd61ed-3423-4999-a306-808b096df282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:00.262640 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd61ed-3423-4999-a306-808b096df282 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:00.263151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd61ed-3423-4999-a306-808b096df282 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:00.263633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd61ed-3423-4999-a306-808b096df282 None admin] PATH_INFO: `/v3/domains/default/users/9fe3a5506b35415a9373071c0b976e34/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:00.266168 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd61ed-3423-4999-a306-808b096df282 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=9fe3a5506b35415a9373071c0b976e34, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:00.267939 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd61ed-3423-4999-a306-808b096df282 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:00.292638 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 19/40] 10.222.0.58 () {58 vars in 1361 bytes} [Thu Sep 1 16:17:00 2022] PUT /identity/v3/domains/default/users/9fe3a5506b35415a9373071c0b976e34/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:17:00.294916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0e3bc-21f9-4763-9e3a-d8437a3dd44a None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:00.295566 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0e3bc-21f9-4763-9e3a-d8437a3dd44a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:00.296312 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0e3bc-21f9-4763-9e3a-d8437a3dd44a None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:00.298171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 20/41] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:00.303379 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0da456e-c105-4a78-a005-86cea00a4905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:00.303911 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da456e-c105-4a78-a005-86cea00a4905 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:00.303911 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da456e-c105-4a78-a005-86cea00a4905 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:00.304237 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da456e-c105-4a78-a005-86cea00a4905 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:00.304846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0da456e-c105-4a78-a005-86cea00a4905 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:00.306761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3629ac-a460-4c23-87f2-08c1ab1b44d6 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:00.307202 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0da456e-c105-4a78-a005-86cea00a4905 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:00.307467 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3629ac-a460-4c23-87f2-08c1ab1b44d6 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:00.307937 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3629ac-a460-4c23-87f2-08c1ab1b44d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:00.331647 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 22/42] 10.222.0.58 () {60 vars in 1131 bytes} [Thu Sep 1 16:17:00 2022] POST /identity/v3/roles => generated 225 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:17:00.332044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5c3629ac-a460-4c23-87f2-08c1ab1b44d6 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:00.333249 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5c3629ac-a460-4c23-87f2-08c1ab1b44d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:00.404955 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0da456e-c105-4a78-a005-86cea00a4905 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:00.424570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 21/43] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:00.554695 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:00.635972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0da456e-c105-4a78-a005-86cea00a4905 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:00.671548 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62caf31d-7854-45be-afad-b5ff8e485326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:00.672171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62caf31d-7854-45be-afad-b5ff8e485326 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:00.672463 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62caf31d-7854-45be-afad-b5ff8e485326 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:00.672742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62caf31d-7854-45be-afad-b5ff8e485326 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:00.673432 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62caf31d-7854-45be-afad-b5ff8e485326 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:00.675027 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62caf31d-7854-45be-afad-b5ff8e485326 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:00.691698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 23/44] 10.222.0.58 () {60 vars in 1131 bytes} [Thu Sep 1 16:17:00 2022] POST /identity/v3/roles => generated 223 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:17:00.702054 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3629ac-a460-4c23-87f2-08c1ab1b44d6 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:00.721950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:00.811699 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3629ac-a460-4c23-87f2-08c1ab1b44d6 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:00.847559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06c6db0-fba8-4d8a-a18e-e6df25319ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:00.848188 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c6db0-fba8-4d8a-a18e-e6df25319ffa None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:00.848465 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c6db0-fba8-4d8a-a18e-e6df25319ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:00.848734 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c6db0-fba8-4d8a-a18e-e6df25319ffa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:00.849808 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06c6db0-fba8-4d8a-a18e-e6df25319ffa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:00.851283 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06c6db0-fba8-4d8a-a18e-e6df25319ffa None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:00.853590 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 22/45] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:17:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:00.858940 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62caf31d-7854-45be-afad-b5ff8e485326 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:00.947644 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:00.963038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62caf31d-7854-45be-afad-b5ff8e485326 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:00.996662 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535d3cc-1fcf-4c95-a4db-09ad613f11a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:00.997278 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535d3cc-1fcf-4c95-a4db-09ad613f11a0 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:00.997565 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535d3cc-1fcf-4c95-a4db-09ad613f11a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:00.997843 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535d3cc-1fcf-4c95-a4db-09ad613f11a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:00.998528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535d3cc-1fcf-4c95-a4db-09ad613f11a0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:00.999980 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535d3cc-1fcf-4c95-a4db-09ad613f11a0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:01.035325 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 24/46] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Sep 1 16:17:00 2022] POST /identity/v3/projects => generated 312 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:17:02.283369 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa507d-1f56-4375-8ef1-de4cdba857e4 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:02.283935 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa507d-1f56-4375-8ef1-de4cdba857e4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:02.283935 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa507d-1f56-4375-8ef1-de4cdba857e4 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:02.285254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 23/47] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:02.290827 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600695a-0592-461a-b2a0-17882dfa7129 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:02.291221 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600695a-0592-461a-b2a0-17882dfa7129 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:02.291396 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600695a-0592-461a-b2a0-17882dfa7129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:02.309893 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0600695a-0592-461a-b2a0-17882dfa7129 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:02.311377 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0600695a-0592-461a-b2a0-17882dfa7129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:02.378280 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 25/48] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:02.649453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa507d-1f56-4375-8ef1-de4cdba857e4 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:02.732266 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:02.748634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa507d-1f56-4375-8ef1-de4cdba857e4 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:02.782725 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a12400-727d-4365-b0b8-50f0315dcbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:02.783358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a12400-727d-4365-b0b8-50f0315dcbb6 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:02.783643 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a12400-727d-4365-b0b8-50f0315dcbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:02.784005 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a12400-727d-4365-b0b8-50f0315dcbb6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:02.795317 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a12400-727d-4365-b0b8-50f0315dcbb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:02.801722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-56a12400-727d-4365-b0b8-50f0315dcbb6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:17:02.801722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:02.801722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:02.801722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:02.801722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:02.801722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:02.801722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:02.801722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:02.801722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:02.801722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:02.801722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:02.801722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:02.801722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:02.801722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:02.801722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:02.801722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:02.802585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:02.802585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:02.802585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:02.802585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:02.802585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:02.802585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:02.802585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:02.802585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:02.802585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:02.802585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:02.802585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:02.802585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:02.802585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:02.802585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:02.802585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:02.802585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:02.803384 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:02.803384 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:02.803384 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:02.803384 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:02.803384 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:02.803384 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:02.803384 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:02.803384 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:02.803384 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:02.803384 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:02.803384 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:17:02.803384 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:17:02.803384 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 24/49] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:17:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:02.808582 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600695a-0592-461a-b2a0-17882dfa7129 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:02.872508 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:02.916679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600695a-0592-461a-b2a0-17882dfa7129 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:02.932858 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c9bfd-4ec2-41fb-a516-1875137fbcf9 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:02.933150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c9bfd-4ec2-41fb-a516-1875137fbcf9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:02.933429 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c9bfd-4ec2-41fb-a516-1875137fbcf9 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:02.934810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 25/50] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:02.939585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e4bc5-3256-407f-9e5f-f2979a4e0c05 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:02.939873 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e4bc5-3256-407f-9e5f-f2979a4e0c05 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:02.940148 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e4bc5-3256-407f-9e5f-f2979a4e0c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:02.950220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d23a3-7cb2-4ab2-b280-980d2add9219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:02.950859 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d23a3-7cb2-4ab2-b280-980d2add9219 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:02.951147 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d23a3-7cb2-4ab2-b280-980d2add9219 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:02.951424 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d23a3-7cb2-4ab2-b280-980d2add9219 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:02.952099 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d23a3-7cb2-4ab2-b280-980d2add9219 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:02.952411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d23a3-7cb2-4ab2-b280-980d2add9219 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:02.953830 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d23a3-7cb2-4ab2-b280-980d2add9219 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:02.958011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5f7e4bc5-3256-407f-9e5f-f2979a4e0c05 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:02.959225 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5f7e4bc5-3256-407f-9e5f-f2979a4e0c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:02.961926 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 26/51] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:02.968188 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d23a3-7cb2-4ab2-b280-980d2add9219 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:03.027608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 26/52] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:17:03.066230 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d23a3-7cb2-4ab2-b280-980d2add9219 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:03.094056 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:03.100357 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fef5e-1816-4560-993e-db9063768c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:03.100993 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fef5e-1816-4560-993e-db9063768c42 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:03.101279 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fef5e-1816-4560-993e-db9063768c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:03.101559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fef5e-1816-4560-993e-db9063768c42 None admin] PATH_INFO: `/v3/users/9fe3a5506b35415a9373071c0b976e34` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:03.102758 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59fef5e-1816-4560-993e-db9063768c42 None admin] RBAC: Authorizing `identity:get_user(user_id=9fe3a5506b35415a9373071c0b976e34)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:03.104842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59fef5e-1816-4560-993e-db9063768c42 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:03.106470 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 27/53] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:02 2022] GET /identity/v3/users/9fe3a5506b35415a9373071c0b976e34 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:03.112035 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e4bc5-3256-407f-9e5f-f2979a4e0c05 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:03.183931 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:03.212174 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e4bc5-3256-407f-9e5f-f2979a4e0c05 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:03.245098 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109658a-a4e0-41da-9d4f-2a3e837b1163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:03.245720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109658a-a4e0-41da-9d4f-2a3e837b1163 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:03.246001 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109658a-a4e0-41da-9d4f-2a3e837b1163 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:03.246303 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109658a-a4e0-41da-9d4f-2a3e837b1163 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:03.247396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109658a-a4e0-41da-9d4f-2a3e837b1163 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:03.248830 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109658a-a4e0-41da-9d4f-2a3e837b1163 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:03.250568 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 27/54] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:17:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:03.256142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fef5e-1816-4560-993e-db9063768c42 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:03.304708 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109658a-a4e0-41da-9d4f-2a3e837b1163 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:03.357927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fef5e-1816-4560-993e-db9063768c42 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:03.392954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74750273-c6d7-483a-8633-1e74df1d7c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:03.393559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74750273-c6d7-483a-8633-1e74df1d7c49 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:03.393842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74750273-c6d7-483a-8633-1e74df1d7c49 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:03.394118 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74750273-c6d7-483a-8633-1e74df1d7c49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:03.394880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74750273-c6d7-483a-8633-1e74df1d7c49 None admin] RBAC: Adding query filter params (role.id=ff293acb8551463aafe6970230042b89, scope.domain.id=default, user.id=9fe3a5506b35415a9373071c0b976e34) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:03.395191 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74750273-c6d7-483a-8633-1e74df1d7c49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:03.396633 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74750273-c6d7-483a-8633-1e74df1d7c49 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:03.402454 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 28/55] 10.222.0.58 () {56 vars in 1322 bytes} [Thu Sep 1 16:17:03 2022] GET /identity/v3/role_assignments?user.id=9fe3a5506b35415a9373071c0b976e34&scope.domain.id=default&role.id=ff293acb8551463aafe6970230042b89 => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:03.407711 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109658a-a4e0-41da-9d4f-2a3e837b1163 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:03.551586 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e987281-ac3b-421c-89e7-18966f6511ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:03.552432 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e987281-ac3b-421c-89e7-18966f6511ae None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:03.552718 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e987281-ac3b-421c-89e7-18966f6511ae None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:03.552994 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e987281-ac3b-421c-89e7-18966f6511ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:03.554108 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e987281-ac3b-421c-89e7-18966f6511ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:03.555592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e987281-ac3b-421c-89e7-18966f6511ae None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:03.558310 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 28/56] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:17:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:03.563383 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74750273-c6d7-483a-8633-1e74df1d7c49 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:03.627952 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:03.667506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74750273-c6d7-483a-8633-1e74df1d7c49 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:03.699227 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e385fc27-b82d-4531-ae1b-514cb31c973c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:03.699831 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e385fc27-b82d-4531-ae1b-514cb31c973c None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:03.700116 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e385fc27-b82d-4531-ae1b-514cb31c973c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:03.700394 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e385fc27-b82d-4531-ae1b-514cb31c973c None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:03.701039 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e385fc27-b82d-4531-ae1b-514cb31c973c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:03.702500 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e385fc27-b82d-4531-ae1b-514cb31c973c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:03.727686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 29/57] 10.222.0.58 () {60 vars in 1140 bytes} [Thu Sep 1 16:17:03 2022] POST /identity/v3/projects => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:17:05.647840 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d30f4-369f-4af1-ad75-a8468a52d240 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:05.648354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d30f4-369f-4af1-ad75-a8468a52d240 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:05.648460 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d30f4-369f-4af1-ad75-a8468a52d240 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:05.650644 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 29/58] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:17:05.655915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518adf6-dd59-437c-8bb6-a47c476a4557 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:05.656308 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518adf6-dd59-437c-8bb6-a47c476a4557 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:05.656465 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518adf6-dd59-437c-8bb6-a47c476a4557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:05.675306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0518adf6-dd59-437c-8bb6-a47c476a4557 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:05.676492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0518adf6-dd59-437c-8bb6-a47c476a4557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:05.739675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 30/59] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:06.016982 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d30f4-369f-4af1-ad75-a8468a52d240 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:06.029607 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:06.121605 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d30f4-369f-4af1-ad75-a8468a52d240 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:06.154593 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f92bce-0652-4dad-96dc-8ddbcdf9a2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:06.155395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f92bce-0652-4dad-96dc-8ddbcdf9a2be None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:06.155701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f92bce-0652-4dad-96dc-8ddbcdf9a2be None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:06.155979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f92bce-0652-4dad-96dc-8ddbcdf9a2be None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:06.157097 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f92bce-0652-4dad-96dc-8ddbcdf9a2be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:06.158583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f92bce-0652-4dad-96dc-8ddbcdf9a2be None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:06.160256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 30/60] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:17:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:06.166395 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518adf6-dd59-437c-8bb6-a47c476a4557 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:06.209041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:06.262533 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518adf6-dd59-437c-8bb6-a47c476a4557 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:06.295539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f973e07-1853-4cad-a2f5-d101ca473c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:06.296115 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f973e07-1853-4cad-a2f5-d101ca473c12 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:06.296400 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f973e07-1853-4cad-a2f5-d101ca473c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:06.296679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f973e07-1853-4cad-a2f5-d101ca473c12 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:06.297354 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f973e07-1853-4cad-a2f5-d101ca473c12 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:06.298863 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f973e07-1853-4cad-a2f5-d101ca473c12 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:06.325601 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 31/61] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Sep 1 16:17:06 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:17:09.284003 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0453804-9d40-4a09-9d72-3a31031b1319 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:09.284500 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0453804-9d40-4a09-9d72-3a31031b1319 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:09.284572 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0453804-9d40-4a09-9d72-3a31031b1319 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:09.286035 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 31/62] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:09.295013 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d1f6e-ccfd-4bdb-9e5d-269f62588a79 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:09.295866 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d1f6e-ccfd-4bdb-9e5d-269f62588a79 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:09.296340 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d1f6e-ccfd-4bdb-9e5d-269f62588a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:09.335284 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-596d1f6e-ccfd-4bdb-9e5d-269f62588a79 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:09.336802 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-596d1f6e-ccfd-4bdb-9e5d-269f62588a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:09.398091 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c14a00-12cd-4ff2-b479-010b4a524fc5 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:09.398764 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c14a00-12cd-4ff2-b479-010b4a524fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:09.399243 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c14a00-12cd-4ff2-b479-010b4a524fc5 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:09.400858 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 32/63] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:09.406003 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5282c-f95c-4cf6-b8c3-ffb76967d4cc None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:09.406581 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5282c-f95c-4cf6-b8c3-ffb76967d4cc None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:09.407071 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5282c-f95c-4cf6-b8c3-ffb76967d4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:09.419350 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 32/64] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:17:09.442372 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5ff5282c-f95c-4cf6-b8c3-ffb76967d4cc None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:09.443575 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5ff5282c-f95c-4cf6-b8c3-ffb76967d4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:09.546585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 33/65] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:09.714515 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d1f6e-ccfd-4bdb-9e5d-269f62588a79 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:09.731299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fb615-fae5-4fc9-b394-3f9fdbbc1830 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:09.731600 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fb615-fae5-4fc9-b394-3f9fdbbc1830 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:09.731870 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fb615-fae5-4fc9-b394-3f9fdbbc1830 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:09.733620 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 34/66] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:09.745125 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a42525-e61c-4a76-a3e2-badba0950bec None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:09.745414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a42525-e61c-4a76-a3e2-badba0950bec None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:09.745694 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a42525-e61c-4a76-a3e2-badba0950bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:09.757466 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:09.768933 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a9a42525-e61c-4a76-a3e2-badba0950bec None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:09.770661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a9a42525-e61c-4a76-a3e2-badba0950bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:09.790617 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:09.857537 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 35/67] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:09.858836 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35e5ec-ebee-4e6d-ad8b-6e6f7dcfdeb8 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:09.859181 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35e5ec-ebee-4e6d-ad8b-6e6f7dcfdeb8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:09.859465 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35e5ec-ebee-4e6d-ad8b-6e6f7dcfdeb8 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:09.860963 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 36/68] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:09.869435 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d1f6e-ccfd-4bdb-9e5d-269f62588a79 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:09.874052 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dac875-895f-45d1-9cea-c0482e841705 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:09.874371 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dac875-895f-45d1-9cea-c0482e841705 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:09.874652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dac875-895f-45d1-9cea-c0482e841705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:09.907707 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316cbcfa-2f0d-472b-a107-7da8be41c506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:09.908441 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316cbcfa-2f0d-472b-a107-7da8be41c506 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:09.908733 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316cbcfa-2f0d-472b-a107-7da8be41c506 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:09.909006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316cbcfa-2f0d-472b-a107-7da8be41c506 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:09.910559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f4dac875-895f-45d1-9cea-c0482e841705 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:09.911773 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f4dac875-895f-45d1-9cea-c0482e841705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:09.921573 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316cbcfa-2f0d-472b-a107-7da8be41c506 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:09.926906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-316cbcfa-2f0d-472b-a107-7da8be41c506 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:09.926906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:09.926906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:09.926906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:09.926906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:09.926906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:09.926906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:09.926906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:09.926906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:09.926906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:09.926906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:09.926906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:09.926906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:09.926906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:09.926906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:09.926906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:09.927757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:09.927757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:09.927757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:09.927757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:09.927757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:09.927757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:09.927757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:09.927757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:09.927757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:09.927757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:09.927757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:09.927757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:09.927757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:09.927757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:09.927757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:09.927757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:09.928545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:09.928545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:09.928545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:09.928545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:09.928545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:09.928545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:09.928545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:09.928545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:09.928545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:09.928545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:09.928545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:09.928545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:17:09.928545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 33/69] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:09.934887 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316cbcfa-2f0d-472b-a107-7da8be41c506 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:10.025047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 37/70] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:10.025921 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dac875-895f-45d1-9cea-c0482e841705 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:10.066557 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:10.079142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316cbcfa-2f0d-472b-a107-7da8be41c506 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:10.108221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:10.125693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424011c0-97c1-47b1-be4a-42e0e0e03625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:10.127119 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424011c0-97c1-47b1-be4a-42e0e0e03625 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:10.127909 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424011c0-97c1-47b1-be4a-42e0e0e03625 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:10.128383 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424011c0-97c1-47b1-be4a-42e0e0e03625 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:10.129506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424011c0-97c1-47b1-be4a-42e0e0e03625 None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:10.130339 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424011c0-97c1-47b1-be4a-42e0e0e03625 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:10.132177 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424011c0-97c1-47b1-be4a-42e0e0e03625 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:10.142932 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 34/71] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Sep 1 16:17:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:10.144216 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c213a-3610-492e-871d-7851d1ba5026 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:10.144690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c213a-3610-492e-871d-7851d1ba5026 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:10.145115 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c213a-3610-492e-871d-7851d1ba5026 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:10.146281 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dac875-895f-45d1-9cea-c0482e841705 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:10.146919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 35/72] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:10.147820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c213a-3610-492e-871d-7851d1ba5026 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:10.187891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf9e1d-f0a6-445a-bf2a-21c44eb195c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:10.188795 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf9e1d-f0a6-445a-bf2a-21c44eb195c2 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:10.189237 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf9e1d-f0a6-445a-bf2a-21c44eb195c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:10.189517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf9e1d-f0a6-445a-bf2a-21c44eb195c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:10.201950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbf9e1d-f0a6-445a-bf2a-21c44eb195c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:10.206926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-3bbf9e1d-f0a6-445a-bf2a-21c44eb195c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:17:10.206926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:10.206926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:10.206926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:10.206926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:10.206926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:10.206926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:10.206926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:10.206926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:10.206926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:10.206926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:10.206926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:10.206926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:10.206926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:10.206926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:10.206926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:10.207770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:10.207770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:10.207770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:10.207770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:10.207770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:10.207770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:10.207770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:10.207770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:10.207770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:10.207770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:10.207770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:10.207770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:10.207770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:10.207770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:10.207770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:10.207770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:10.208548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:10.208548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:10.208548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:10.208548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:10.208548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:10.208548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:10.208548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:10.208548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:10.208548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:10.208548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:10.208548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:17:10.208548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:17:10.208548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 38/73] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:17:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:10.210034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf9e1d-f0a6-445a-bf2a-21c44eb195c2 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:10.267352 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c213a-3610-492e-871d-7851d1ba5026 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:10.302771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6560279f-6407-4d7d-b156-baecc90578b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:10.303414 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560279f-6407-4d7d-b156-baecc90578b7 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:10.303604 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560279f-6407-4d7d-b156-baecc90578b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:10.303887 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560279f-6407-4d7d-b156-baecc90578b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:10.311563 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf9e1d-f0a6-445a-bf2a-21c44eb195c2 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:10.315350 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6560279f-6407-4d7d-b156-baecc90578b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:10.320345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-6560279f-6407-4d7d-b156-baecc90578b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:10.320345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:10.320345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:10.320345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:10.320345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:10.320345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:10.320345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:10.320345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:10.320345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:10.320345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:10.320345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:10.320345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:10.320345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:10.320345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:10.320345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:10.320345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:10.321205 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:10.321205 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:10.321205 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:10.321205 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:10.321205 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:10.321205 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:10.321205 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:10.321205 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:10.321205 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:10.321205 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:10.321205 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:10.321205 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:10.321205 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:10.321205 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:10.321205 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:10.321205 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:10.322546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:10.322546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:10.322546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:10.322546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:10.322546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:10.322546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:10.322546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:10.322546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:10.322546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:10.322546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:10.322546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:10.322546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:17:10.322546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 36/74] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:10.323179 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046e2f8-237f-4c90-b1bd-b33be0fce519 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:10.323847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046e2f8-237f-4c90-b1bd-b33be0fce519 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:10.323847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046e2f8-237f-4c90-b1bd-b33be0fce519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:10.341557 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b046e2f8-237f-4c90-b1bd-b33be0fce519 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:10.342835 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b046e2f8-237f-4c90-b1bd-b33be0fce519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:10.342973 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:10.350225 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:10.351056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95bfb64-05f5-48e1-baf7-58a8f91dc910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:10.351736 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bfb64-05f5-48e1-baf7-58a8f91dc910 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:10.352017 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bfb64-05f5-48e1-baf7-58a8f91dc910 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:10.352291 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bfb64-05f5-48e1-baf7-58a8f91dc910 None admin] PATH_INFO: `/v3/users/3c9bc22bd8da47e98a53e9c6e78236ae` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:10.367097 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95bfb64-05f5-48e1-baf7-58a8f91dc910 None admin] RBAC: Authorizing `identity:get_user(user_id=3c9bc22bd8da47e98a53e9c6e78236ae)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:10.369076 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95bfb64-05f5-48e1-baf7-58a8f91dc910 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:10.370801 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 39/75] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:10 2022] GET /identity/v3/users/3c9bc22bd8da47e98a53e9c6e78236ae => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:10.375628 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95bfb64-05f5-48e1-baf7-58a8f91dc910 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:10.409900 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 37/76] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:10.418378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046e2f8-237f-4c90-b1bd-b33be0fce519 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:10.473218 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95bfb64-05f5-48e1-baf7-58a8f91dc910 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:10.507479 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a531190-953d-488f-a34e-f97178fb26bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:10.508081 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a531190-953d-488f-a34e-f97178fb26bc None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:10.508366 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a531190-953d-488f-a34e-f97178fb26bc None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:10.508646 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a531190-953d-488f-a34e-f97178fb26bc None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:10.509397 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a531190-953d-488f-a34e-f97178fb26bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:10.509708 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a531190-953d-488f-a34e-f97178fb26bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:10.509848 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046e2f8-237f-4c90-b1bd-b33be0fce519 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:10.511102 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a531190-953d-488f-a34e-f97178fb26bc None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:10.519720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 40/77] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:10.520341 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a531190-953d-488f-a34e-f97178fb26bc None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:10.549173 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e436b-1b2a-4b6b-9059-9d3573ef7215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:10.549811 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e436b-1b2a-4b6b-9059-9d3573ef7215 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:10.550093 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e436b-1b2a-4b6b-9059-9d3573ef7215 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:10.550406 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e436b-1b2a-4b6b-9059-9d3573ef7215 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:10.561169 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e436b-1b2a-4b6b-9059-9d3573ef7215 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:10.565963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-163e436b-1b2a-4b6b-9059-9d3573ef7215 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:17:10.565963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:10.565963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:10.565963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:10.565963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:10.565963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:10.565963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:10.565963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:10.565963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:10.565963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:10.565963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:10.565963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:10.565963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:10.565963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:10.565963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:10.565963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:10.566875 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:10.566875 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:10.566875 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:10.566875 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:10.566875 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:10.566875 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:10.566875 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:10.566875 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:10.566875 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:10.566875 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:10.566875 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:10.566875 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:10.566875 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:10.566875 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:10.566875 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:10.566875 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:10.571030 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:10.571030 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:10.571030 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:10.571030 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:10.571030 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:10.571030 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:10.571030 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:10.571030 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:10.571030 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:10.571030 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:10.571030 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:17:10.571030 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:17:10.571030 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 38/78] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Sep 1 16:17:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:10.571030 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e436b-1b2a-4b6b-9059-9d3573ef7215 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:10.574367 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:10.579493 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:10.632543 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a531190-953d-488f-a34e-f97178fb26bc None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:10.668959 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed3207-2ade-495a-8aa3-5817f9a6cde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:10.669578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed3207-2ade-495a-8aa3-5817f9a6cde6 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:10.669865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed3207-2ade-495a-8aa3-5817f9a6cde6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:10.670144 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed3207-2ade-495a-8aa3-5817f9a6cde6 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:10.670852 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed3207-2ade-495a-8aa3-5817f9a6cde6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:10.671160 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed3207-2ade-495a-8aa3-5817f9a6cde6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:10.672546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed3207-2ade-495a-8aa3-5817f9a6cde6 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:10.675452 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e436b-1b2a-4b6b-9059-9d3573ef7215 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:10.681869 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 41/79] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Sep 1 16:17:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:10.682725 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed3207-2ade-495a-8aa3-5817f9a6cde6 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:10.709579 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf789d-c5be-4381-ade4-e924bf463bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:10.710184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf789d-c5be-4381-ade4-e924bf463bda None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:10.710490 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf789d-c5be-4381-ade4-e924bf463bda None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:10.710767 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf789d-c5be-4381-ade4-e924bf463bda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:10.777503 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf789d-c5be-4381-ade4-e924bf463bda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:10.779023 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf789d-c5be-4381-ade4-e924bf463bda None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:10.780813 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:10.781532 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed3207-2ade-495a-8aa3-5817f9a6cde6 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:10.793137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:10.815978 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb680551-c56c-4091-ad4a-bd9857377ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:10.816535 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb680551-c56c-4091-ad4a-bd9857377ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:10.816845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb680551-c56c-4091-ad4a-bd9857377ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:10.817119 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb680551-c56c-4091-ad4a-bd9857377ca9 None admin] PATH_INFO: `/v3/users/9fe3a5506b35415a9373071c0b976e34` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:10.818272 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb680551-c56c-4091-ad4a-bd9857377ca9 None admin] RBAC: Authorizing `identity:get_user(user_id=9fe3a5506b35415a9373071c0b976e34)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:10.819711 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb680551-c56c-4091-ad4a-bd9857377ca9 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:10.821265 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 42/80] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:10 2022] GET /identity/v3/users/9fe3a5506b35415a9373071c0b976e34 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:10.822536 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb680551-c56c-4091-ad4a-bd9857377ca9 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:10.877808 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 39/81] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:17:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:17:10.882599 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf789d-c5be-4381-ade4-e924bf463bda None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:10.925134 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb680551-c56c-4091-ad4a-bd9857377ca9 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:10.958827 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ba1ea-86d5-4711-a20c-6ace12b8be92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:10.959431 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ba1ea-86d5-4711-a20c-6ace12b8be92 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:10.959713 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ba1ea-86d5-4711-a20c-6ace12b8be92 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:10.959987 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ba1ea-86d5-4711-a20c-6ace12b8be92 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:10.960653 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ba1ea-86d5-4711-a20c-6ace12b8be92 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:10.960985 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ba1ea-86d5-4711-a20c-6ace12b8be92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:10.962437 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ba1ea-86d5-4711-a20c-6ace12b8be92 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:10.971043 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 43/82] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:17:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:10.971745 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ba1ea-86d5-4711-a20c-6ace12b8be92 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:10.984846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf789d-c5be-4381-ade4-e924bf463bda None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:10.986543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:11.014423 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:11.020924 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce66814-1b28-463e-871e-c0f4471d3e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:11.021556 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce66814-1b28-463e-871e-c0f4471d3e3c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:11.021844 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce66814-1b28-463e-871e-c0f4471d3e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:11.022241 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce66814-1b28-463e-871e-c0f4471d3e3c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:11.023285 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce66814-1b28-463e-871e-c0f4471d3e3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:11.024807 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce66814-1b28-463e-871e-c0f4471d3e3c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:11.027047 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 40/83] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:17:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:11.032331 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce66814-1b28-463e-871e-c0f4471d3e3c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:11.075249 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ba1ea-86d5-4711-a20c-6ace12b8be92 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:11.107779 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498cc4c-9238-4065-a47b-1fa2d1795e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:11.108401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498cc4c-9238-4065-a47b-1fa2d1795e02 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:11.108684 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498cc4c-9238-4065-a47b-1fa2d1795e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:11.108956 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498cc4c-9238-4065-a47b-1fa2d1795e02 None admin] PATH_INFO: `/v3/users/3c9bc22bd8da47e98a53e9c6e78236ae` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:11.110083 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f498cc4c-9238-4065-a47b-1fa2d1795e02 None admin] RBAC: Authorizing `identity:get_user(user_id=3c9bc22bd8da47e98a53e9c6e78236ae)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:11.111538 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f498cc4c-9238-4065-a47b-1fa2d1795e02 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:11.113104 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 44/84] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:10 2022] GET /identity/v3/users/3c9bc22bd8da47e98a53e9c6e78236ae => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:11.114987 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498cc4c-9238-4065-a47b-1fa2d1795e02 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:11.137086 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce66814-1b28-463e-871e-c0f4471d3e3c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:11.171504 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90307fd5-275c-4397-9632-b1d6fb331f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:11.172121 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90307fd5-275c-4397-9632-b1d6fb331f11 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:11.172410 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90307fd5-275c-4397-9632-b1d6fb331f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:11.172689 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90307fd5-275c-4397-9632-b1d6fb331f11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:11.210482 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:11.211886 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498cc4c-9238-4065-a47b-1fa2d1795e02 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:11.221251 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:11.240411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90307fd5-275c-4397-9632-b1d6fb331f11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:11.241880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90307fd5-275c-4397-9632-b1d6fb331f11 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:11.247006 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8bd34-70a0-4fda-be40-b4deeaf64409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:11.247636 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8bd34-70a0-4fda-be40-b4deeaf64409 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:11.247918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8bd34-70a0-4fda-be40-b4deeaf64409 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:11.248195 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8bd34-70a0-4fda-be40-b4deeaf64409 None admin] PATH_INFO: `/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:11.249351 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d8bd34-70a0-4fda-be40-b4deeaf64409 None admin] RBAC: Authorizing `identity:get_project(project_id=9ea4cc94eaf84d0c8fec375d056242eb)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:11.250932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d8bd34-70a0-4fda-be40-b4deeaf64409 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:11.252614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 45/85] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:11 2022] GET /identity/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:11.258337 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8bd34-70a0-4fda-be40-b4deeaf64409 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:11.342820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 41/86] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:17:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:17:11.346998 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90307fd5-275c-4397-9632-b1d6fb331f11 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:11.355006 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8bd34-70a0-4fda-be40-b4deeaf64409 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:11.388087 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a1b89-cd81-4ac8-901d-5ba3031f6c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:11.388729 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a1b89-cd81-4ac8-901d-5ba3031f6c8c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:11.389012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a1b89-cd81-4ac8-901d-5ba3031f6c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:11.389288 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a1b89-cd81-4ac8-901d-5ba3031f6c8c None admin] PATH_INFO: `/v3/users/3c9bc22bd8da47e98a53e9c6e78236ae` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:11.390472 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a1b89-cd81-4ac8-901d-5ba3031f6c8c None admin] RBAC: Authorizing `identity:get_user(user_id=3c9bc22bd8da47e98a53e9c6e78236ae)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:11.391909 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a1b89-cd81-4ac8-901d-5ba3031f6c8c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:11.393497 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 46/87] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:11 2022] GET /identity/v3/users/3c9bc22bd8da47e98a53e9c6e78236ae => generated 275 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:11.394324 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a1b89-cd81-4ac8-901d-5ba3031f6c8c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:11.416331 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:11.443567 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:11.446878 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90307fd5-275c-4397-9632-b1d6fb331f11 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:11.479982 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4193f-f6f1-4c82-a746-ea9898e2196d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:11.480587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4193f-f6f1-4c82-a746-ea9898e2196d None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:11.480872 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4193f-f6f1-4c82-a746-ea9898e2196d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:11.481147 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4193f-f6f1-4c82-a746-ea9898e2196d None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:11.481802 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4193f-f6f1-4c82-a746-ea9898e2196d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:11.483289 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4193f-f6f1-4c82-a746-ea9898e2196d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:11.497189 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a1b89-cd81-4ac8-901d-5ba3031f6c8c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:11.504691 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 42/88] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Sep 1 16:17:11 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:17:11.505507 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4193f-f6f1-4c82-a746-ea9898e2196d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:11.534266 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5235a1-92b4-41c5-a3a2-04c82adc4e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:11.534929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5235a1-92b4-41c5-a3a2-04c82adc4e76 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:11.535174 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5235a1-92b4-41c5-a3a2-04c82adc4e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:11.535468 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5235a1-92b4-41c5-a3a2-04c82adc4e76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:11.611118 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5235a1-92b4-41c5-a3a2-04c82adc4e76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:11.612630 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5235a1-92b4-41c5-a3a2-04c82adc4e76 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:11.633141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4193f-f6f1-4c82-a746-ea9898e2196d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:11.663401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:11.667770 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:11.679707 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6679344-32f3-4cbf-b794-f45dac6a3347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:11.680322 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6679344-32f3-4cbf-b794-f45dac6a3347 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:11.680609 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6679344-32f3-4cbf-b794-f45dac6a3347 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:11.680888 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6679344-32f3-4cbf-b794-f45dac6a3347 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:11.681693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6679344-32f3-4cbf-b794-f45dac6a3347 None admin] RBAC: Adding query filter params (role.id=42db5f955f2644b1bb22b0a68f8cd829, scope.project.id=9ea4cc94eaf84d0c8fec375d056242eb, user.id=3c9bc22bd8da47e98a53e9c6e78236ae) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:11.682013 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6679344-32f3-4cbf-b794-f45dac6a3347 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:11.683545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6679344-32f3-4cbf-b794-f45dac6a3347 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:11.690167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 43/89] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:17:11 2022] GET /identity/v3/role_assignments?user.id=3c9bc22bd8da47e98a53e9c6e78236ae&scope.project.id=9ea4cc94eaf84d0c8fec375d056242eb&role.id=42db5f955f2644b1bb22b0a68f8cd829 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:11.696093 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6679344-32f3-4cbf-b794-f45dac6a3347 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:11.744581 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 47/90] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:17:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:17:11.745613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5235a1-92b4-41c5-a3a2-04c82adc4e76 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:11.828988 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6679344-32f3-4cbf-b794-f45dac6a3347 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:11.861695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5235a1-92b4-41c5-a3a2-04c82adc4e76 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:11.871104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8470c-9620-4efc-a6c4-a00f7809e9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:11.871798 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8470c-9620-4efc-a6c4-a00f7809e9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:11.872092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8470c-9620-4efc-a6c4-a00f7809e9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:11.872372 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8470c-9620-4efc-a6c4-a00f7809e9a9 None admin] PATH_INFO: `/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:11.874223 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8470c-9620-4efc-a6c4-a00f7809e9a9 None admin] RBAC: Authorizing `identity:get_project(project_id=9ea4cc94eaf84d0c8fec375d056242eb)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:11.875697 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8470c-9620-4efc-a6c4-a00f7809e9a9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:11.877362 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 44/91] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:11 2022] GET /identity/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:11.878147 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8470c-9620-4efc-a6c4-a00f7809e9a9 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:11.906078 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87caa900-5f23-4654-9400-a85747a7e520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:11.907352 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87caa900-5f23-4654-9400-a85747a7e520 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:11.907817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87caa900-5f23-4654-9400-a85747a7e520 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:11.908292 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87caa900-5f23-4654-9400-a85747a7e520 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:11.928691 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:11.950593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:11.994316 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87caa900-5f23-4654-9400-a85747a7e520 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:11.995793 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87caa900-5f23-4654-9400-a85747a7e520 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:12.007117 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8470c-9620-4efc-a6c4-a00f7809e9a9 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:12.053475 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf862b00-2a15-4ba5-89ab-5e0f48da0962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:12.055181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf862b00-2a15-4ba5-89ab-5e0f48da0962 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:12.055836 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf862b00-2a15-4ba5-89ab-5e0f48da0962 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:12.057972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf862b00-2a15-4ba5-89ab-5e0f48da0962 None admin] PATH_INFO: `/v3/projects/34b5f0c9ed7a4c86a6c36d4534ba0d55` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:12.060232 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf862b00-2a15-4ba5-89ab-5e0f48da0962 None admin] RBAC: Authorizing `identity:get_project(project_id=34b5f0c9ed7a4c86a6c36d4534ba0d55)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:12.062087 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf862b00-2a15-4ba5-89ab-5e0f48da0962 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:12.067394 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 45/92] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:11 2022] GET /identity/v3/projects/34b5f0c9ed7a4c86a6c36d4534ba0d55 => generated 312 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:12.069703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf862b00-2a15-4ba5-89ab-5e0f48da0962 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:12.140451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 48/93] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:17:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:17:12.142591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87caa900-5f23-4654-9400-a85747a7e520 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:12.231786 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:12.260194 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf862b00-2a15-4ba5-89ab-5e0f48da0962 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:12.315221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87caa900-5f23-4654-9400-a85747a7e520 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:12.317182 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:12.352543 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc949e3a-ab67-4190-8050-3c15f40b03a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:12.354045 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc949e3a-ab67-4190-8050-3c15f40b03a4 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:12.354482 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9febc1f-2269-426d-871d-017dcdc59103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:12.355115 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9febc1f-2269-426d-871d-017dcdc59103 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:12.355383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc949e3a-ab67-4190-8050-3c15f40b03a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:12.355753 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9febc1f-2269-426d-871d-017dcdc59103 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:12.355753 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9febc1f-2269-426d-871d-017dcdc59103 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:12.356300 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc949e3a-ab67-4190-8050-3c15f40b03a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:12.356401 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9febc1f-2269-426d-871d-017dcdc59103 None admin] RBAC: Adding query filter params (role.id=ff293acb8551463aafe6970230042b89, scope.project.id=9ea4cc94eaf84d0c8fec375d056242eb, user.id=9fe3a5506b35415a9373071c0b976e34) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:12.356725 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9febc1f-2269-426d-871d-017dcdc59103 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:12.357535 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc949e3a-ab67-4190-8050-3c15f40b03a4 None admin] RBAC: Adding query filter params (role.id=42db5f955f2644b1bb22b0a68f8cd829, scope.project.id=34b5f0c9ed7a4c86a6c36d4534ba0d55, user.id=3c9bc22bd8da47e98a53e9c6e78236ae) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:12.358036 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc949e3a-ab67-4190-8050-3c15f40b03a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:12.358235 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9febc1f-2269-426d-871d-017dcdc59103 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:12.359860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc949e3a-ab67-4190-8050-3c15f40b03a4 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:12.368372 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 49/94] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:17:12 2022] GET /identity/v3/role_assignments?user.id=3c9bc22bd8da47e98a53e9c6e78236ae&scope.project.id=34b5f0c9ed7a4c86a6c36d4534ba0d55&role.id=42db5f955f2644b1bb22b0a68f8cd829 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:12.370122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc949e3a-ab67-4190-8050-3c15f40b03a4 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:12.384683 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 46/95] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:17:12 2022] GET /identity/v3/role_assignments?user.id=9fe3a5506b35415a9373071c0b976e34&scope.project.id=9ea4cc94eaf84d0c8fec375d056242eb&role.id=ff293acb8551463aafe6970230042b89 => generated 259 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:12.529044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc949e3a-ab67-4190-8050-3c15f40b03a4 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:12.544273 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:12.576737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6cb967-c80f-4cdf-8f11-b73ff479d215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:12.577405 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6cb967-c80f-4cdf-8f11-b73ff479d215 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:12.577710 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6cb967-c80f-4cdf-8f11-b73ff479d215 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:12.577979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6cb967-c80f-4cdf-8f11-b73ff479d215 None admin] PATH_INFO: `/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:12.579235 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6cb967-c80f-4cdf-8f11-b73ff479d215 None admin] RBAC: Authorizing `identity:get_project(project_id=9ea4cc94eaf84d0c8fec375d056242eb)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:12.580716 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6cb967-c80f-4cdf-8f11-b73ff479d215 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:12.582767 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 50/96] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:12 2022] GET /identity/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:12.588589 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9febc1f-2269-426d-871d-017dcdc59103 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:12.714878 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9febc1f-2269-426d-871d-017dcdc59103 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:12.756238 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb5d9a-5319-4e4e-9032-dce21c668e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:12.756877 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb5d9a-5319-4e4e-9032-dce21c668e74 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:12.757192 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb5d9a-5319-4e4e-9032-dce21c668e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:12.757656 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb5d9a-5319-4e4e-9032-dce21c668e74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:12.758432 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bb5d9a-5319-4e4e-9032-dce21c668e74 None admin] RBAC: Adding query filter params (role.id=72700efc336a4b4f992b08ff9ea99f0c, scope.project.id=9ea4cc94eaf84d0c8fec375d056242eb, user.id=3c9bc22bd8da47e98a53e9c6e78236ae) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:12.758755 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bb5d9a-5319-4e4e-9032-dce21c668e74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:12.760269 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bb5d9a-5319-4e4e-9032-dce21c668e74 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:12.765947 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 47/97] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:17:12 2022] GET /identity/v3/role_assignments?user.id=3c9bc22bd8da47e98a53e9c6e78236ae&scope.project.id=9ea4cc94eaf84d0c8fec375d056242eb&role.id=72700efc336a4b4f992b08ff9ea99f0c => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:14.028654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b934f1-4aad-45f0-8a22-62d58af1d6b5 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:14.029163 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b934f1-4aad-45f0-8a22-62d58af1d6b5 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:14.029238 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b934f1-4aad-45f0-8a22-62d58af1d6b5 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:14.030844 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 51/98] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:14.039366 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b0afe-17b4-4d2e-b125-2b34629481b2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:14.039761 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b0afe-17b4-4d2e-b125-2b34629481b2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:14.039927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b0afe-17b4-4d2e-b125-2b34629481b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:14.067202 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-261b0afe-17b4-4d2e-b125-2b34629481b2 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:14.068436 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-261b0afe-17b4-4d2e-b125-2b34629481b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:14.136770 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:14.160074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 48/99] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:14.462267 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1d0be-6290-4cd5-8372-3f60c9f85d1b None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:14.462267 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1d0be-6290-4cd5-8372-3f60c9f85d1b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:14.462267 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1d0be-6290-4cd5-8372-3f60c9f85d1b None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:14.463451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 52/100] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:14.471860 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97985e6-5633-4875-8454-8b9b64397fe2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:14.472147 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97985e6-5633-4875-8454-8b9b64397fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:14.472425 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97985e6-5633-4875-8454-8b9b64397fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:14.495031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c97985e6-5633-4875-8454-8b9b64397fe2 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:14.496217 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c97985e6-5633-4875-8454-8b9b64397fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:14.585118 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 49/101] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:14.732854 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610289ee-79c4-47b0-8fda-9cb627741f6a None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:14.733143 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610289ee-79c4-47b0-8fda-9cb627741f6a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:14.733335 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610289ee-79c4-47b0-8fda-9cb627741f6a None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:14.734956 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 53/102] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:14.735974 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97985e6-5633-4875-8454-8b9b64397fe2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:14.739839 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7319d-4762-4472-82de-0eb2213c08dd None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:14.740111 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7319d-4762-4472-82de-0eb2213c08dd None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:14.740387 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7319d-4762-4472-82de-0eb2213c08dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:14.776295 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-02d7319d-4762-4472-82de-0eb2213c08dd None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:14.777460 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-02d7319d-4762-4472-82de-0eb2213c08dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:14.907414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 54/103] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:14.952577 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97985e6-5633-4875-8454-8b9b64397fe2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:14.975321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:14.999978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006297fd-79e7-489a-8473-4a9d215eca55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:15.000942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006297fd-79e7-489a-8473-4a9d215eca55 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:15.001440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006297fd-79e7-489a-8473-4a9d215eca55 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:15.001914 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006297fd-79e7-489a-8473-4a9d215eca55 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:15.019089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006297fd-79e7-489a-8473-4a9d215eca55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:15.024333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d06f66-c6a4-41f0-8047-77f048ed7906 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:15.024626 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d06f66-c6a4-41f0-8047-77f048ed7906 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:15.024895 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-006297fd-79e7-489a-8473-4a9d215eca55 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:17:15.024895 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:15.024895 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:15.024895 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:15.024895 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:15.024895 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:15.024895 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:15.024895 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:15.024895 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:15.024895 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:15.024895 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:15.024895 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:15.024895 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:15.024895 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:15.024895 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:15.024895 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:15.025724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:15.025724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:15.025724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:15.025724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:15.025724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:15.025724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:15.025724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:15.025724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:15.025724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:15.025724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:15.025724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:15.025724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:15.025724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:15.025724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:15.025724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:15.025724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:15.026950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:15.026950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:15.026950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:15.026950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:15.026950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:15.026950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:15.026950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:15.026950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:15.026950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:15.026950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:15.026950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:17:15.026950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:17:15.027760 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d06f66-c6a4-41f0-8047-77f048ed7906 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:15.027760 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 55/104] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:17:15.029187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 50/105] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:15.034018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be897f12-f688-40ff-a609-9fa9f8decb4a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:15.034482 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be897f12-f688-40ff-a609-9fa9f8decb4a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:15.034665 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be897f12-f688-40ff-a609-9fa9f8decb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:15.040041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006297fd-79e7-489a-8473-4a9d215eca55 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:15.066952 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-be897f12-f688-40ff-a609-9fa9f8decb4a None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:15.068161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-be897f12-f688-40ff-a609-9fa9f8decb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:15.190262 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 56/106] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:15.191126 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be897f12-f688-40ff-a609-9fa9f8decb4a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:15.226689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:15.252561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006297fd-79e7-489a-8473-4a9d215eca55 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:15.298659 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be897f12-f688-40ff-a609-9fa9f8decb4a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:15.346230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c9c8b-93f6-4a3a-b4c3-38083adb9b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:15.347284 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c9c8b-93f6-4a3a-b4c3-38083adb9b1d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:15.347717 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c9c8b-93f6-4a3a-b4c3-38083adb9b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:15.348138 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c9c8b-93f6-4a3a-b4c3-38083adb9b1d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:15.358911 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8e5b6-c190-40b4-8267-7057413f86ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:15.359539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8e5b6-c190-40b4-8267-7057413f86ad None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:15.359824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8e5b6-c190-40b4-8267-7057413f86ad None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:15.360101 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8e5b6-c190-40b4-8267-7057413f86ad None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:15.360785 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8e5b6-c190-40b4-8267-7057413f86ad None admin] RBAC: Adding query filter params (name=reader) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:15.361090 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8e5b6-c190-40b4-8267-7057413f86ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:15.362501 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8e5b6-c190-40b4-8267-7057413f86ad None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:15.364213 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c9c8b-93f6-4a3a-b4c3-38083adb9b1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:15.373981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-9e5c9c8b-93f6-4a3a-b4c3-38083adb9b1d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:15.373981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:15.373981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:15.373981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:15.373981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:15.373981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:15.373981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:15.373981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:15.373981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:15.373981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:15.373981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:15.373981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:15.373981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:15.373981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:15.373981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:15.373981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:15.374904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:15.374904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:15.374904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:15.374904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:15.374904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:15.374904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:15.374904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:15.374904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:15.374904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:15.374904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:15.374904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:15.374904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:15.374904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:15.374904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:15.374904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:15.374904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:15.376153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:15.376153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:15.376153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:15.376153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:15.376153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:15.376153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:15.376153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:15.376153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:15.376153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:15.376153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:15.376153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:15.376153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:17:15.376153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 57/107] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:15.376153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c9c8b-93f6-4a3a-b4c3-38083adb9b1d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:15.377777 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 51/108] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Sep 1 16:17:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:15.391513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8e5b6-c190-40b4-8267-7057413f86ad None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:15.441817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:15.491067 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:15.498310 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c9c8b-93f6-4a3a-b4c3-38083adb9b1d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:15.512356 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8e5b6-c190-40b4-8267-7057413f86ad None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:15.533038 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d891806-9b04-4107-9622-aaada5e90eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:15.533721 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d891806-9b04-4107-9622-aaada5e90eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:15.534006 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d891806-9b04-4107-9622-aaada5e90eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:15.534332 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d891806-9b04-4107-9622-aaada5e90eb5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:15.535460 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d891806-9b04-4107-9622-aaada5e90eb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:15.536916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d891806-9b04-4107-9622-aaada5e90eb5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:15.538616 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 58/109] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:17:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:15.539397 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d891806-9b04-4107-9622-aaada5e90eb5 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:15.553151 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae8c5f-fe91-401d-bbf7-654a6bf515fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:15.553896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae8c5f-fe91-401d-bbf7-654a6bf515fe None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:15.554187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae8c5f-fe91-401d-bbf7-654a6bf515fe None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:15.554554 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae8c5f-fe91-401d-bbf7-654a6bf515fe None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:15.555402 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae8c5f-fe91-401d-bbf7-654a6bf515fe None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:15.555722 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae8c5f-fe91-401d-bbf7-654a6bf515fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:15.557902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae8c5f-fe91-401d-bbf7-654a6bf515fe None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:15.568973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 52/110] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Sep 1 16:17:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:15.570310 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b55459-4a76-435d-b116-39ca5d6256c0 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:15.570606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b55459-4a76-435d-b116-39ca5d6256c0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:15.570887 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b55459-4a76-435d-b116-39ca5d6256c0 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:15.572850 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 53/111] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:15.573441 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b55459-4a76-435d-b116-39ca5d6256c0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:15.651816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d891806-9b04-4107-9622-aaada5e90eb5 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:15.680762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:15.685754 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b55459-4a76-435d-b116-39ca5d6256c0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:15.701561 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552598a0-024a-48a9-9e1b-5cfe014d2264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:15.702283 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552598a0-024a-48a9-9e1b-5cfe014d2264 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:15.702568 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552598a0-024a-48a9-9e1b-5cfe014d2264 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:15.702839 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552598a0-024a-48a9-9e1b-5cfe014d2264 None admin] PATH_INFO: `/v3/users/ed0ffdea22f44c0f9c9ddb3ba52a22db` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:15.720285 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552598a0-024a-48a9-9e1b-5cfe014d2264 None admin] RBAC: Authorizing `identity:get_user(user_id=ed0ffdea22f44c0f9c9ddb3ba52a22db)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:15.721760 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552598a0-024a-48a9-9e1b-5cfe014d2264 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:15.723482 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 59/112] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:15 2022] GET /identity/v3/users/ed0ffdea22f44c0f9c9ddb3ba52a22db => generated 289 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:15.727280 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552598a0-024a-48a9-9e1b-5cfe014d2264 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:15.730110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7d306-c6be-426b-ae37-f04bb9c6d8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:15.730757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7d306-c6be-426b-ae37-f04bb9c6d8dd None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:15.731040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7d306-c6be-426b-ae37-f04bb9c6d8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:15.731315 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7d306-c6be-426b-ae37-f04bb9c6d8dd None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:15.731965 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7d306-c6be-426b-ae37-f04bb9c6d8dd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:15.733416 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7d306-c6be-426b-ae37-f04bb9c6d8dd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:15.754626 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:15.760720 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 54/113] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Sep 1 16:17:15 2022] POST /identity/v3/projects => generated 302 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:17:15.761556 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7d306-c6be-426b-ae37-f04bb9c6d8dd None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:15.843216 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552598a0-024a-48a9-9e1b-5cfe014d2264 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:15.880140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7d306-c6be-426b-ae37-f04bb9c6d8dd None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:15.882160 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d169026-b3d1-4d54-bfc3-3761feaed86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:15.882982 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d169026-b3d1-4d54-bfc3-3761feaed86f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:15.883336 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d169026-b3d1-4d54-bfc3-3761feaed86f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:15.883678 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d169026-b3d1-4d54-bfc3-3761feaed86f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:15.895741 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d169026-b3d1-4d54-bfc3-3761feaed86f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:15.900835 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-9d169026-b3d1-4d54-bfc3-3761feaed86f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:17:15.900835 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:15.900835 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:15.900835 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:15.900835 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:15.900835 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:15.900835 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:15.900835 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:15.900835 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:15.900835 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:15.900835 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:15.900835 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:15.900835 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:15.900835 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:15.900835 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:15.900835 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:15.901707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:15.901707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:15.901707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:15.901707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:15.901707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:15.901707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:15.901707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:15.901707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:15.901707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:15.901707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:15.901707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:15.901707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:15.901707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:15.901707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:15.901707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:15.901707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:15.902720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:15.902720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:15.902720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:15.902720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:15.902720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:15.902720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:15.902720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:15.902720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:15.902720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:15.902720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:15.902720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:17:15.902720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:17:15.903343 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 60/114] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:17:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:15.904287 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180ab5a-4448-4115-893c-f4d6956da322 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:15.904381 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180ab5a-4448-4115-893c-f4d6956da322 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:15.904655 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180ab5a-4448-4115-893c-f4d6956da322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:15.922039 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773d1ad-4e1e-441c-9a45-6fb420d51d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:15.922728 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773d1ad-4e1e-441c-9a45-6fb420d51d76 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:15.923008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773d1ad-4e1e-441c-9a45-6fb420d51d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:15.923285 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773d1ad-4e1e-441c-9a45-6fb420d51d76 None admin] PATH_INFO: `/v3/users/3c9bc22bd8da47e98a53e9c6e78236ae` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:15.924419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0773d1ad-4e1e-441c-9a45-6fb420d51d76 None admin] RBAC: Authorizing `identity:get_user(user_id=3c9bc22bd8da47e98a53e9c6e78236ae)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:15.925878 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0773d1ad-4e1e-441c-9a45-6fb420d51d76 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:15.927041 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7180ab5a-4448-4115-893c-f4d6956da322 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:15.927771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 55/115] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:15 2022] GET /identity/v3/users/3c9bc22bd8da47e98a53e9c6e78236ae => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:15.928778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7180ab5a-4448-4115-893c-f4d6956da322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:15.928908 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773d1ad-4e1e-441c-9a45-6fb420d51d76 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:15.956521 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:16.006644 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 61/116] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:16.007896 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e0f9b-57cd-45fd-85fb-3edfc7b8bd09 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:16.008176 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e0f9b-57cd-45fd-85fb-3edfc7b8bd09 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:16.008447 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e0f9b-57cd-45fd-85fb-3edfc7b8bd09 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:16.010265 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 62/117] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:16.010961 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e0f9b-57cd-45fd-85fb-3edfc7b8bd09 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:16.039255 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:16.042812 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773d1ad-4e1e-441c-9a45-6fb420d51d76 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:16.086972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608cd7ae-1870-4f1f-b0d1-47e8b3cd9e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:16.088023 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cd7ae-1870-4f1f-b0d1-47e8b3cd9e07 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:16.088559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cd7ae-1870-4f1f-b0d1-47e8b3cd9e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:16.089069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cd7ae-1870-4f1f-b0d1-47e8b3cd9e07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:16.166743 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e0f9b-57cd-45fd-85fb-3edfc7b8bd09 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:16.170159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608cd7ae-1870-4f1f-b0d1-47e8b3cd9e07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:16.171864 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608cd7ae-1870-4f1f-b0d1-47e8b3cd9e07 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:16.191503 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:16.215850 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7db1a-38ff-485b-9be0-0546169bcc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:16.216467 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7db1a-38ff-485b-9be0-0546169bcc62 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:16.216759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7db1a-38ff-485b-9be0-0546169bcc62 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:16.217087 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7db1a-38ff-485b-9be0-0546169bcc62 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:16.218313 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c7db1a-38ff-485b-9be0-0546169bcc62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:16.218679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c7db1a-38ff-485b-9be0-0546169bcc62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:16.220632 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c7db1a-38ff-485b-9be0-0546169bcc62 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:16.234342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 63/118] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:16.235525 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7db1a-38ff-485b-9be0-0546169bcc62 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:16.291831 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 56/119] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:17:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:17:16.293128 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4753620-3a3f-4e36-8763-b15fc2790986 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:16.293430 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4753620-3a3f-4e36-8763-b15fc2790986 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:16.293715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4753620-3a3f-4e36-8763-b15fc2790986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:16.312485 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e4753620-3a3f-4e36-8763-b15fc2790986 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:16.313703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e4753620-3a3f-4e36-8763-b15fc2790986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:16.332930 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:16.361148 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7db1a-38ff-485b-9be0-0546169bcc62 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:16.387151 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 57/120] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:16.388200 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4753620-3a3f-4e36-8763-b15fc2790986 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:16.410310 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7addf1-c52a-4f90-b857-bb7aa628a214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:16.410952 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7addf1-c52a-4f90-b857-bb7aa628a214 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:16.411296 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7addf1-c52a-4f90-b857-bb7aa628a214 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:16.411580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7addf1-c52a-4f90-b857-bb7aa628a214 None admin] PATH_INFO: `/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:16.412895 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7addf1-c52a-4f90-b857-bb7aa628a214 None admin] RBAC: Authorizing `identity:get_project(project_id=9ea4cc94eaf84d0c8fec375d056242eb)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:16.414562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7addf1-c52a-4f90-b857-bb7aa628a214 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:16.422229 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 64/121] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:16 2022] GET /identity/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:16.423016 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7addf1-c52a-4f90-b857-bb7aa628a214 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:16.451689 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:16.519275 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4753620-3a3f-4e36-8763-b15fc2790986 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:16.535075 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7addf1-c52a-4f90-b857-bb7aa628a214 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:16.567182 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ad004-4212-40d0-8667-68a02d603e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:16.568215 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ad004-4212-40d0-8667-68a02d603e44 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:16.568573 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ad004-4212-40d0-8667-68a02d603e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:16.569359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ad004-4212-40d0-8667-68a02d603e44 None admin] PATH_INFO: `/v3/users/9fe3a5506b35415a9373071c0b976e34` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:16.571041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24ad004-4212-40d0-8667-68a02d603e44 None admin] RBAC: Authorizing `identity:get_user(user_id=9fe3a5506b35415a9373071c0b976e34)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:16.573047 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24ad004-4212-40d0-8667-68a02d603e44 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:16.574171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d54051-6506-4c00-89e0-6852078b8f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:16.574897 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 58/122] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:16 2022] GET /identity/v3/users/9fe3a5506b35415a9373071c0b976e34 => generated 247 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:16.575209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d54051-6506-4c00-89e0-6852078b8f37 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:16.575635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d54051-6506-4c00-89e0-6852078b8f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:16.575838 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ad004-4212-40d0-8667-68a02d603e44 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:16.576307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d54051-6506-4c00-89e0-6852078b8f37 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:16.588202 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d54051-6506-4c00-89e0-6852078b8f37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:16.593613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-51d54051-6506-4c00-89e0-6852078b8f37 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:17:16.593613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:16.593613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:16.593613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:16.593613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:16.593613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:16.593613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:16.593613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:16.593613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:16.593613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:16.593613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:16.593613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:16.593613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:16.593613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:16.593613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:16.593613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:16.594689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:16.594689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:16.594689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:16.594689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:16.594689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:16.594689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:16.594689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:16.594689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:16.594689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:16.594689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:16.594689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:16.594689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:16.594689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:16.594689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:16.594689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:16.594689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:16.595641 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:16.595641 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:16.595641 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:16.595641 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:16.595641 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:16.595641 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:16.595641 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:16.595641 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:16.595641 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:16.595641 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:16.595641 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:17:16.595641 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:17:16.602568 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 65/123] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Sep 1 16:17:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:16.604237 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d54051-6506-4c00-89e0-6852078b8f37 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:16.608184 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:16.705132 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ad004-4212-40d0-8667-68a02d603e44 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:16.714242 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d54051-6506-4c00-89e0-6852078b8f37 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:16.721642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:16.740504 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b947fdf-9633-4b9f-89d2-b13beb6801ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:16.741167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b947fdf-9633-4b9f-89d2-b13beb6801ea None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:16.741449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b947fdf-9633-4b9f-89d2-b13beb6801ea None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:16.741725 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b947fdf-9633-4b9f-89d2-b13beb6801ea None admin] PATH_INFO: `/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:16.742876 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b947fdf-9633-4b9f-89d2-b13beb6801ea None admin] RBAC: Authorizing `identity:get_project(project_id=9ea4cc94eaf84d0c8fec375d056242eb)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:16.744860 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b947fdf-9633-4b9f-89d2-b13beb6801ea None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:16.746511 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 59/124] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:16 2022] GET /identity/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:16.747268 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b947fdf-9633-4b9f-89d2-b13beb6801ea None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:16.757635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a37bc6-7d1e-4e59-94b2-0f396d8390aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:16.758592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a37bc6-7d1e-4e59-94b2-0f396d8390aa None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:16.759057 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a37bc6-7d1e-4e59-94b2-0f396d8390aa None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:16.760196 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a37bc6-7d1e-4e59-94b2-0f396d8390aa None admin] PATH_INFO: `/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb/users/3c9bc22bd8da47e98a53e9c6e78236ae/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:16.762557 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a37bc6-7d1e-4e59-94b2-0f396d8390aa None admin] RBAC: Authorizing `identity:create_grant(project_id=9ea4cc94eaf84d0c8fec375d056242eb, user_id=3c9bc22bd8da47e98a53e9c6e78236ae, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:16.764193 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a37bc6-7d1e-4e59-94b2-0f396d8390aa None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:16.786229 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 66/125] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Sep 1 16:17:16 2022] PUT /identity/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb/users/3c9bc22bd8da47e98a53e9c6e78236ae/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:17:16.786229 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a37bc6-7d1e-4e59-94b2-0f396d8390aa None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:16.866159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b947fdf-9633-4b9f-89d2-b13beb6801ea None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:16.877592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:16.919371 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88601aa7-1ac8-4b0f-868b-0c8ad9a73fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:16.920271 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88601aa7-1ac8-4b0f-868b-0c8ad9a73fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:16.920970 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88601aa7-1ac8-4b0f-868b-0c8ad9a73fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:16.921894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88601aa7-1ac8-4b0f-868b-0c8ad9a73fc2 None admin] PATH_INFO: `/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:16.923572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88601aa7-1ac8-4b0f-868b-0c8ad9a73fc2 None admin] RBAC: Authorizing `identity:get_project(project_id=9ea4cc94eaf84d0c8fec375d056242eb)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:16.926285 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88601aa7-1ac8-4b0f-868b-0c8ad9a73fc2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:16.929641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 60/126] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:16 2022] GET /identity/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:16.930622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88601aa7-1ac8-4b0f-868b-0c8ad9a73fc2 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:16.931776 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a37bc6-7d1e-4e59-94b2-0f396d8390aa None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:16.973877 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71eb6d-4f66-4bae-ada7-c3a69db8e14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:16.974447 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71eb6d-4f66-4bae-ada7-c3a69db8e14c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:16.974733 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71eb6d-4f66-4bae-ada7-c3a69db8e14c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:16.975011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71eb6d-4f66-4bae-ada7-c3a69db8e14c None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:16.975694 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71eb6d-4f66-4bae-ada7-c3a69db8e14c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:16.976000 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71eb6d-4f66-4bae-ada7-c3a69db8e14c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:16.977417 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71eb6d-4f66-4bae-ada7-c3a69db8e14c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:16.988737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 67/127] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:17:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:16.989965 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71eb6d-4f66-4bae-ada7-c3a69db8e14c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:17.005167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:17.051759 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88601aa7-1ac8-4b0f-868b-0c8ad9a73fc2 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:17.094917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a58efc2-ef71-49b7-b390-cf27f846312c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:17.095891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58efc2-ef71-49b7-b390-cf27f846312c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:17.096393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58efc2-ef71-49b7-b390-cf27f846312c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:17.096810 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58efc2-ef71-49b7-b390-cf27f846312c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:17.109765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a58efc2-ef71-49b7-b390-cf27f846312c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:17.111261 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71eb6d-4f66-4bae-ada7-c3a69db8e14c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:17.115762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-7a58efc2-ef71-49b7-b390-cf27f846312c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:17.115762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:17.115762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:17.115762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:17.115762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:17.115762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:17.115762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:17.115762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:17.115762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:17.115762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:17.115762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:17.115762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:17.115762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:17.115762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:17.115762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:17.115762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:17.116669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:17.116669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:17.116669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:17.116669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:17.116669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:17.116669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:17.116669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:17.116669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:17.116669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:17.116669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:17.116669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:17.116669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:17.116669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:17.116669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:17.116669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:17.116669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:17.117460 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:17.117460 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:17.117460 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:17.117460 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:17.117460 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:17.117460 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:17.117460 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:17.117460 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:17.117460 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:17.117460 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:17.117460 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:17.117460 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:17:17.119883 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 61/128] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:17.125870 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a58efc2-ef71-49b7-b390-cf27f846312c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:17.135990 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:17.153164 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a9be9a-0f23-4d19-a768-1aabcc979908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:17.153782 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9be9a-0f23-4d19-a768-1aabcc979908 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:17.154068 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9be9a-0f23-4d19-a768-1aabcc979908 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:17.154399 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9be9a-0f23-4d19-a768-1aabcc979908 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:17.155278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a9be9a-0f23-4d19-a768-1aabcc979908 None admin] RBAC: Adding query filter params (role.id=d657364ebb9c4110bc1acfe8a9f2f3fd, scope.project.id=9ea4cc94eaf84d0c8fec375d056242eb, user.id=ed0ffdea22f44c0f9c9ddb3ba52a22db) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:17.156061 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a9be9a-0f23-4d19-a768-1aabcc979908 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:17.157834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a9be9a-0f23-4d19-a768-1aabcc979908 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:17.174226 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 68/129] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:17:16 2022] GET /identity/v3/role_assignments?user.id=ed0ffdea22f44c0f9c9ddb3ba52a22db&scope.project.id=9ea4cc94eaf84d0c8fec375d056242eb&role.id=d657364ebb9c4110bc1acfe8a9f2f3fd => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:17.174226 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a9be9a-0f23-4d19-a768-1aabcc979908 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:17.292036 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a9be9a-0f23-4d19-a768-1aabcc979908 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:17.337736 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1cf9a1-ea9c-428a-be03-c86bbc47b39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:17.338414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cf9a1-ea9c-428a-be03-c86bbc47b39e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:17.338697 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cf9a1-ea9c-428a-be03-c86bbc47b39e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:17.338972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cf9a1-ea9c-428a-be03-c86bbc47b39e None admin] PATH_INFO: `/v3/users/3c9bc22bd8da47e98a53e9c6e78236ae` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:17.341626 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a58efc2-ef71-49b7-b390-cf27f846312c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:17.343903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1cf9a1-ea9c-428a-be03-c86bbc47b39e None admin] RBAC: Authorizing `identity:get_user(user_id=3c9bc22bd8da47e98a53e9c6e78236ae)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:17.345388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1cf9a1-ea9c-428a-be03-c86bbc47b39e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:17.347129 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 69/130] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:17 2022] GET /identity/v3/users/3c9bc22bd8da47e98a53e9c6e78236ae => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:17.348037 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1cf9a1-ea9c-428a-be03-c86bbc47b39e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:17.372122 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:17.425540 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:17.427578 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368df73-6ff8-4fc5-b845-6c00ddbf4aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:17.428964 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368df73-6ff8-4fc5-b845-6c00ddbf4aa7 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:17.429843 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368df73-6ff8-4fc5-b845-6c00ddbf4aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:17.432329 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368df73-6ff8-4fc5-b845-6c00ddbf4aa7 None admin] PATH_INFO: `/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb/users/9fe3a5506b35415a9373071c0b976e34/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:17.435700 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4368df73-6ff8-4fc5-b845-6c00ddbf4aa7 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ea4cc94eaf84d0c8fec375d056242eb, user_id=9fe3a5506b35415a9373071c0b976e34, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:17.437718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4368df73-6ff8-4fc5-b845-6c00ddbf4aa7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:17.475426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 62/131] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Sep 1 16:17:17 2022] PUT /identity/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb/users/9fe3a5506b35415a9373071c0b976e34/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:17:17.476595 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368df73-6ff8-4fc5-b845-6c00ddbf4aa7 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:17.498239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1cf9a1-ea9c-428a-be03-c86bbc47b39e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:17.534233 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badacf78-3e28-4c85-8623-919c3ecfa152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:17.534233 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badacf78-3e28-4c85-8623-919c3ecfa152 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:17.534233 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badacf78-3e28-4c85-8623-919c3ecfa152 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:17.534233 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badacf78-3e28-4c85-8623-919c3ecfa152 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:17.534633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badacf78-3e28-4c85-8623-919c3ecfa152 None admin] RBAC: Adding query filter params (name=member) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:17.534633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badacf78-3e28-4c85-8623-919c3ecfa152 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:17.534949 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badacf78-3e28-4c85-8623-919c3ecfa152 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:17.547624 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 70/132] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Sep 1 16:17:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:17.557329 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badacf78-3e28-4c85-8623-919c3ecfa152 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:17.679326 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badacf78-3e28-4c85-8623-919c3ecfa152 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:17.700092 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:17.720938 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c815f6f4-749d-47f3-bcf8-7e84f745dff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:17.721726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815f6f4-749d-47f3-bcf8-7e84f745dff0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:17.722080 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815f6f4-749d-47f3-bcf8-7e84f745dff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:17.722522 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815f6f4-749d-47f3-bcf8-7e84f745dff0 None admin] PATH_INFO: `/v3/users/3c9bc22bd8da47e98a53e9c6e78236ae` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:17.724026 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c815f6f4-749d-47f3-bcf8-7e84f745dff0 None admin] RBAC: Authorizing `identity:get_user(user_id=3c9bc22bd8da47e98a53e9c6e78236ae)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:17.725987 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c815f6f4-749d-47f3-bcf8-7e84f745dff0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:17.727688 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 71/133] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:17 2022] GET /identity/v3/users/3c9bc22bd8da47e98a53e9c6e78236ae => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:17.734572 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c815f6f4-749d-47f3-bcf8-7e84f745dff0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:17.741249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368df73-6ff8-4fc5-b845-6c00ddbf4aa7 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:17.808023 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa66d75-c765-43bd-8f47-0970ea6eb19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:17.808624 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa66d75-c765-43bd-8f47-0970ea6eb19a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:17.808906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa66d75-c765-43bd-8f47-0970ea6eb19a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:17.809186 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa66d75-c765-43bd-8f47-0970ea6eb19a None admin] PATH_INFO: `/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:17.814965 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa66d75-c765-43bd-8f47-0970ea6eb19a None admin] RBAC: Authorizing `identity:get_project(project_id=9ea4cc94eaf84d0c8fec375d056242eb)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:17.816459 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa66d75-c765-43bd-8f47-0970ea6eb19a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:17.818182 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 63/134] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:17 2022] GET /identity/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb => generated 298 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:17.824438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa66d75-c765-43bd-8f47-0970ea6eb19a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:17.845147 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c815f6f4-749d-47f3-bcf8-7e84f745dff0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:17.882279 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23070107-41a4-487d-b480-ca1cd7e04ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:17.882968 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23070107-41a4-487d-b480-ca1cd7e04ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:17.883253 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23070107-41a4-487d-b480-ca1cd7e04ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:17.883530 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23070107-41a4-487d-b480-ca1cd7e04ac5 None admin] PATH_INFO: `/v3/projects/34b5f0c9ed7a4c86a6c36d4534ba0d55` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:17.884759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23070107-41a4-487d-b480-ca1cd7e04ac5 None admin] RBAC: Authorizing `identity:get_project(project_id=34b5f0c9ed7a4c86a6c36d4534ba0d55)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:17.886363 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23070107-41a4-487d-b480-ca1cd7e04ac5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:17.888082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 72/135] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:17 2022] GET /identity/v3/projects/34b5f0c9ed7a4c86a6c36d4534ba0d55 => generated 312 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:17.889194 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136a402-cf68-4ba7-98b6-1f1b17bd7430 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:17.889478 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136a402-cf68-4ba7-98b6-1f1b17bd7430 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:17.889757 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136a402-cf68-4ba7-98b6-1f1b17bd7430 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:17.895210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 73/136] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:17.896702 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e136a402-cf68-4ba7-98b6-1f1b17bd7430 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:17.926564 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:17.958735 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:17.967353 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa66d75-c765-43bd-8f47-0970ea6eb19a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:18.004063 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06232a8b-5cf2-4248-a47c-728d7dcd789c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:18.004748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06232a8b-5cf2-4248-a47c-728d7dcd789c None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:18.005044 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06232a8b-5cf2-4248-a47c-728d7dcd789c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:18.005335 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06232a8b-5cf2-4248-a47c-728d7dcd789c None admin] PATH_INFO: `/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb/users/3c9bc22bd8da47e98a53e9c6e78236ae/roles/72700efc336a4b4f992b08ff9ea99f0c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:18.007577 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e136a402-cf68-4ba7-98b6-1f1b17bd7430 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:18.007697 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06232a8b-5cf2-4248-a47c-728d7dcd789c None admin] RBAC: Authorizing `identity:create_grant(project_id=9ea4cc94eaf84d0c8fec375d056242eb, user_id=3c9bc22bd8da47e98a53e9c6e78236ae, role_id=72700efc336a4b4f992b08ff9ea99f0c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:18.009797 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06232a8b-5cf2-4248-a47c-728d7dcd789c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:18.030439 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 64/137] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Sep 1 16:17:17 2022] PUT /identity/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb/users/3c9bc22bd8da47e98a53e9c6e78236ae/roles/72700efc336a4b4f992b08ff9ea99f0c => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:17:18.031916 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e2501-18cc-49cc-a0ed-9feb214ba2b7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:18.032208 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e2501-18cc-49cc-a0ed-9feb214ba2b7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:18.032487 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e2501-18cc-49cc-a0ed-9feb214ba2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:18.047945 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a0b09-73ed-4769-b43f-155f852379ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:18.047945 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a0b09-73ed-4769-b43f-155f852379ce None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:18.048314 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a0b09-73ed-4769-b43f-155f852379ce None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:18.049369 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a0b09-73ed-4769-b43f-155f852379ce None admin] PATH_INFO: `/v3/projects/34b5f0c9ed7a4c86a6c36d4534ba0d55/users/3c9bc22bd8da47e98a53e9c6e78236ae/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:18.050878 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a0b09-73ed-4769-b43f-155f852379ce None admin] RBAC: Authorizing `identity:create_grant(project_id=34b5f0c9ed7a4c86a6c36d4534ba0d55, user_id=3c9bc22bd8da47e98a53e9c6e78236ae, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:18.052730 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a0b09-73ed-4769-b43f-155f852379ce None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:18.079583 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-949e2501-18cc-49cc-a0ed-9feb214ba2b7 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:18.081486 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 74/138] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Sep 1 16:17:17 2022] PUT /identity/v3/projects/34b5f0c9ed7a4c86a6c36d4534ba0d55/users/3c9bc22bd8da47e98a53e9c6e78236ae/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:17:18.081637 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-949e2501-18cc-49cc-a0ed-9feb214ba2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:18.184481 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 65/139] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:18.538815 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a0b09-73ed-4769-b43f-155f852379ce None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:18.660318 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a0b09-73ed-4769-b43f-155f852379ce None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:18.704365 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e665eb-6ee2-44b6-9f42-fbeadd514fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:18.705182 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:18.705671 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e665eb-6ee2-44b6-9f42-fbeadd514fc7 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:18.705970 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e665eb-6ee2-44b6-9f42-fbeadd514fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:18.706268 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e665eb-6ee2-44b6-9f42-fbeadd514fc7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:18.709847 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e665eb-6ee2-44b6-9f42-fbeadd514fc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:18.711329 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e665eb-6ee2-44b6-9f42-fbeadd514fc7 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:18.713411 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 75/140] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:17:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:18.720135 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e2501-18cc-49cc-a0ed-9feb214ba2b7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:18.762660 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:18.855179 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e2501-18cc-49cc-a0ed-9feb214ba2b7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:18.891771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d2d61-f743-4624-967a-291324d38fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:18.892817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d2d61-f743-4624-967a-291324d38fff None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:18.893300 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d2d61-f743-4624-967a-291324d38fff None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:18.893770 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d2d61-f743-4624-967a-291324d38fff None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:18.894686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72d2d61-f743-4624-967a-291324d38fff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:18.896389 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72d2d61-f743-4624-967a-291324d38fff None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:18.921776 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 66/141] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Sep 1 16:17:18 2022] POST /identity/v3/users => generated 283 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:17:19.237437 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfe01c-dd76-4ae2-8c78-4ddb2df03913 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:19.237840 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfe01c-dd76-4ae2-8c78-4ddb2df03913 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:19.238012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfe01c-dd76-4ae2-8c78-4ddb2df03913 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:19.239570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 76/142] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:19.245660 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77579fa7-f055-4be0-b584-3b1303fdc8e5 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:19.246514 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77579fa7-f055-4be0-b584-3b1303fdc8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:19.246995 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77579fa7-f055-4be0-b584-3b1303fdc8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:19.275403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-77579fa7-f055-4be0-b584-3b1303fdc8e5 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:19.276925 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-77579fa7-f055-4be0-b584-3b1303fdc8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:19.355955 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 67/143] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:19.632765 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfe01c-dd76-4ae2-8c78-4ddb2df03913 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:19.764739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfe01c-dd76-4ae2-8c78-4ddb2df03913 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:19.817838 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7c9f4-1d4d-4bcf-9f11-80a89339b42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:19.819225 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7c9f4-1d4d-4bcf-9f11-80a89339b42d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:19.819932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7c9f4-1d4d-4bcf-9f11-80a89339b42d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:19.820704 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7c9f4-1d4d-4bcf-9f11-80a89339b42d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:19.839399 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7c9f4-1d4d-4bcf-9f11-80a89339b42d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:19.847880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-bbf7c9f4-1d4d-4bcf-9f11-80a89339b42d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:19.847880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:19.847880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:19.847880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:19.847880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:19.847880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:19.847880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:19.847880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:19.847880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:19.847880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:19.847880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:19.847880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:19.847880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:19.847880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:19.847880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:19.847880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:19.848837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:19.848837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:19.848837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:19.848837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:19.848837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:19.848837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:19.848837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:19.848837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:19.848837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:19.848837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:19.848837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:19.848837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:19.848837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:19.848837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:19.848837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:19.848837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:19.849634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:19.849634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:19.849634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:19.849634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:19.849634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:19.849634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:19.849634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:19.849634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:19.849634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:19.849634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:19.849634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:19.849634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:17:19.852682 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 77/144] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:19.861406 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77579fa7-f055-4be0-b584-3b1303fdc8e5 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:19.887733 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:19.986655 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77579fa7-f055-4be0-b584-3b1303fdc8e5 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:20.027708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04f1de-837e-464d-9dda-996b8ea8a56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:20.028766 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04f1de-837e-464d-9dda-996b8ea8a56b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:20.029253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04f1de-837e-464d-9dda-996b8ea8a56b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:20.029735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04f1de-837e-464d-9dda-996b8ea8a56b None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:20.030675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04f1de-837e-464d-9dda-996b8ea8a56b None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:20.031236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04f1de-837e-464d-9dda-996b8ea8a56b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:20.032970 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04f1de-837e-464d-9dda-996b8ea8a56b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:20.043914 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 68/145] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Sep 1 16:17:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:20.052091 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7c9f4-1d4d-4bcf-9f11-80a89339b42d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:20.098796 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17c4d0-6be4-4c45-8d64-95a827f750b5 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:20.099437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17c4d0-6be4-4c45-8d64-95a827f750b5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:20.099937 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17c4d0-6be4-4c45-8d64-95a827f750b5 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:20.101549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 69/146] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:20.109990 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:20.112171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8f6a6-f1fd-46be-b101-8b17f292338b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:20.112867 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8f6a6-f1fd-46be-b101-8b17f292338b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:20.113320 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8f6a6-f1fd-46be-b101-8b17f292338b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:20.145349 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c1e8f6a6-f1fd-46be-b101-8b17f292338b None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:20.146493 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c1e8f6a6-f1fd-46be-b101-8b17f292338b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:20.239896 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7c9f4-1d4d-4bcf-9f11-80a89339b42d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:20.288992 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 70/147] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:17:20.296052 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f2b9a-a72e-4621-9dbb-78b74362336a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:20.296690 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f2b9a-a72e-4621-9dbb-78b74362336a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:20.296972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f2b9a-a72e-4621-9dbb-78b74362336a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:20.297247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f2b9a-a72e-4621-9dbb-78b74362336a None admin] PATH_INFO: `/v3/users/3c9bc22bd8da47e98a53e9c6e78236ae` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:20.298413 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442f2b9a-a72e-4621-9dbb-78b74362336a None admin] RBAC: Authorizing `identity:get_user(user_id=3c9bc22bd8da47e98a53e9c6e78236ae)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:20.299915 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442f2b9a-a72e-4621-9dbb-78b74362336a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:20.301771 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 78/148] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:20 2022] GET /identity/v3/users/3c9bc22bd8da47e98a53e9c6e78236ae => generated 275 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:20.309529 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8f6a6-f1fd-46be-b101-8b17f292338b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:20.320878 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:20.429493 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8f6a6-f1fd-46be-b101-8b17f292338b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:20.473460 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03cee37-f9dd-43fd-8e28-6d6956d6a8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:20.474079 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03cee37-f9dd-43fd-8e28-6d6956d6a8e3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:20.474384 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03cee37-f9dd-43fd-8e28-6d6956d6a8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:20.475335 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03cee37-f9dd-43fd-8e28-6d6956d6a8e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:20.550560 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03cee37-f9dd-43fd-8e28-6d6956d6a8e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:20.552111 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03cee37-f9dd-43fd-8e28-6d6956d6a8e3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:20.562385 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:20.677345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 71/149] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:17:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:17:20.685061 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f2b9a-a72e-4621-9dbb-78b74362336a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:20.700020 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a59bc9-627e-4c9d-9e58-fe8fb75bbd71 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:20.700393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a59bc9-627e-4c9d-9e58-fe8fb75bbd71 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:20.700543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a59bc9-627e-4c9d-9e58-fe8fb75bbd71 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:20.701902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 72/150] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:20.708020 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee32ce-57ea-434b-9c89-e882b9b67f44 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:20.708297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee32ce-57ea-434b-9c89-e882b9b67f44 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:20.708571 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee32ce-57ea-434b-9c89-e882b9b67f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:20.732527 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fdee32ce-57ea-434b-9c89-e882b9b67f44 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:20.733723 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fdee32ce-57ea-434b-9c89-e882b9b67f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:20.836925 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 73/151] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:20.890380 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f2b9a-a72e-4621-9dbb-78b74362336a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:20.905818 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:20.936396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15017e35-dedd-49d5-99c2-778b1ae770ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:20.936960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15017e35-dedd-49d5-99c2-778b1ae770ed None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:20.937244 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15017e35-dedd-49d5-99c2-778b1ae770ed None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:20.937522 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15017e35-dedd-49d5-99c2-778b1ae770ed None admin] PATH_INFO: `/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:20.938689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15017e35-dedd-49d5-99c2-778b1ae770ed None admin] RBAC: Authorizing `identity:get_project(project_id=9ea4cc94eaf84d0c8fec375d056242eb)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:20.940155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15017e35-dedd-49d5-99c2-778b1ae770ed None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:20.942878 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 79/152] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:20 2022] GET /identity/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:20.948080 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee32ce-57ea-434b-9c89-e882b9b67f44 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:20.950471 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15017e35-dedd-49d5-99c2-778b1ae770ed None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:21.024003 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:21.111917 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15017e35-dedd-49d5-99c2-778b1ae770ed None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:21.159795 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d30c4-ced9-402a-bc3d-7015b65306c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:21.160438 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d30c4-ced9-402a-bc3d-7015b65306c9 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:21.160693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee32ce-57ea-434b-9c89-e882b9b67f44 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:21.160804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d30c4-ced9-402a-bc3d-7015b65306c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:21.160999 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d30c4-ced9-402a-bc3d-7015b65306c9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:21.183662 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d30c4-ced9-402a-bc3d-7015b65306c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:21.189025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-331d30c4-ced9-402a-bc3d-7015b65306c9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:17:21.189025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:21.189025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:21.189025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:21.189025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:21.189025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:21.189025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:21.189025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:21.189025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:21.189025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:21.189025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:21.189025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:21.189025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:21.189025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:21.189025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:21.189025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:21.190011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:21.190011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:21.190011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:21.190011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:21.190011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:21.190011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:21.190011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:21.190011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:21.190011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:21.190011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:21.190011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:21.190011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:21.190011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:21.190011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:21.190011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:21.190011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:21.190876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:21.190876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:21.190876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:21.190876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:21.190876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:21.190876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:21.190876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:21.190876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:21.190876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:21.190876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:21.190876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:17:21.190876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:17:21.190876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 80/153] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:17:21.195931 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d30c4-ced9-402a-bc3d-7015b65306c9 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:21.224896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5286d71-a094-4e04-8ab6-9eae13f4cd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:21.225873 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5286d71-a094-4e04-8ab6-9eae13f4cd1b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:21.226387 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5286d71-a094-4e04-8ab6-9eae13f4cd1b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:21.226907 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5286d71-a094-4e04-8ab6-9eae13f4cd1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:21.228913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5286d71-a094-4e04-8ab6-9eae13f4cd1b None admin] RBAC: Adding query filter params (role.id=42db5f955f2644b1bb22b0a68f8cd829, scope.project.id=9ea4cc94eaf84d0c8fec375d056242eb, user.id=3c9bc22bd8da47e98a53e9c6e78236ae) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:21.229442 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5286d71-a094-4e04-8ab6-9eae13f4cd1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:21.231173 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5286d71-a094-4e04-8ab6-9eae13f4cd1b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:21.243827 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 74/154] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:17:20 2022] GET /identity/v3/role_assignments?user.id=3c9bc22bd8da47e98a53e9c6e78236ae&scope.project.id=9ea4cc94eaf84d0c8fec375d056242eb&role.id=42db5f955f2644b1bb22b0a68f8cd829 => generated 608 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:21.261772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:21.315679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5286d71-a094-4e04-8ab6-9eae13f4cd1b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:21.341788 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d30c4-ced9-402a-bc3d-7015b65306c9 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:21.400151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bdb493-016c-4011-9dd4-e6c4c174c084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:21.400783 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdb493-016c-4011-9dd4-e6c4c174c084 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:21.401065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdb493-016c-4011-9dd4-e6c4c174c084 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:21.401335 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdb493-016c-4011-9dd4-e6c4c174c084 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:21.402026 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bdb493-016c-4011-9dd4-e6c4c174c084 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:21.402361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bdb493-016c-4011-9dd4-e6c4c174c084 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:21.403798 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bdb493-016c-4011-9dd4-e6c4c174c084 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:21.425620 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 81/155] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Sep 1 16:17:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:21.426936 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49d621-719a-4ff3-8a8c-68822762703d None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:21.427224 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49d621-719a-4ff3-8a8c-68822762703d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:21.427496 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49d621-719a-4ff3-8a8c-68822762703d None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:21.428705 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 82/156] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:21.438079 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72075b6-356d-4b9b-84f6-dca380f2b038 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:21.438434 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72075b6-356d-4b9b-84f6-dca380f2b038 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:21.438678 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72075b6-356d-4b9b-84f6-dca380f2b038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:21.473273 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e72075b6-356d-4b9b-84f6-dca380f2b038 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:21.474460 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e72075b6-356d-4b9b-84f6-dca380f2b038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:21.561240 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5286d71-a094-4e04-8ab6-9eae13f4cd1b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:21.567032 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:21.589502 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 83/157] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:21.590388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72075b6-356d-4b9b-84f6-dca380f2b038 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:21.636539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da02c47-4177-47d7-ae10-07c127ebd3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:21.637183 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da02c47-4177-47d7-ae10-07c127ebd3a9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:21.637465 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da02c47-4177-47d7-ae10-07c127ebd3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:21.637738 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da02c47-4177-47d7-ae10-07c127ebd3a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:21.661893 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:21.667014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da02c47-4177-47d7-ae10-07c127ebd3a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:21.673079 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-7da02c47-4177-47d7-ae10-07c127ebd3a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:17:21.673079 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:21.673079 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:21.673079 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:21.673079 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:21.673079 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:21.673079 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:21.673079 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:21.673079 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:21.673079 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:21.673079 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:21.673079 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:21.673079 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:21.673079 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:21.673079 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:21.673079 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:21.674048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:21.674048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:21.674048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:21.674048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:21.674048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:21.674048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:21.674048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:21.674048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:21.674048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:21.674048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:21.674048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:21.674048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:21.674048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:21.674048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:21.674048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:21.674048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:21.675003 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:21.675003 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:21.675003 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:21.675003 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:21.675003 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:21.675003 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:21.675003 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:21.675003 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:21.675003 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:21.675003 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:21.675003 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:17:21.675003 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:17:21.683929 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 75/158] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:17:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:17:21.684911 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da02c47-4177-47d7-ae10-07c127ebd3a9 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:21.809544 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72075b6-356d-4b9b-84f6-dca380f2b038 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:21.881341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da02c47-4177-47d7-ae10-07c127ebd3a9 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:21.926220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84631880-b0a7-49ea-a2bc-0333366adf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:21.926220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84631880-b0a7-49ea-a2bc-0333366adf2a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:21.926220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84631880-b0a7-49ea-a2bc-0333366adf2a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:21.926220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84631880-b0a7-49ea-a2bc-0333366adf2a None admin] PATH_INFO: `/v3/users/ed0ffdea22f44c0f9c9ddb3ba52a22db` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:21.926617 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84631880-b0a7-49ea-a2bc-0333366adf2a None admin] RBAC: Authorizing `identity:get_user(user_id=ed0ffdea22f44c0f9c9ddb3ba52a22db)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:21.926617 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84631880-b0a7-49ea-a2bc-0333366adf2a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:21.926617 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 84/159] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:21 2022] GET /identity/v3/users/ed0ffdea22f44c0f9c9ddb3ba52a22db => generated 289 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:21.928003 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf60fc0-5f84-4687-921a-2f8ef50ba5b3 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:21.928524 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf60fc0-5f84-4687-921a-2f8ef50ba5b3 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:21.928635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf60fc0-5f84-4687-921a-2f8ef50ba5b3 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:21.930027 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 85/160] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:21.934987 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec0f80-7b77-45ba-ac9d-7889a6a9da10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:21.935128 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf60fc0-5f84-4687-921a-2f8ef50ba5b3 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:21.935617 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec0f80-7b77-45ba-ac9d-7889a6a9da10 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:21.935900 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec0f80-7b77-45ba-ac9d-7889a6a9da10 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:21.936175 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec0f80-7b77-45ba-ac9d-7889a6a9da10 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:21.936861 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec0f80-7b77-45ba-ac9d-7889a6a9da10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:21.937166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec0f80-7b77-45ba-ac9d-7889a6a9da10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:21.938624 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec0f80-7b77-45ba-ac9d-7889a6a9da10 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:21.959994 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 76/161] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:21.961370 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd284e-cd4f-464e-a8e6-b0368f72fd00 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:21.961633 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd284e-cd4f-464e-a8e6-b0368f72fd00 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:21.961915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd284e-cd4f-464e-a8e6-b0368f72fd00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:21.996885 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9cfd284e-cd4f-464e-a8e6-b0368f72fd00 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:21.998368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9cfd284e-cd4f-464e-a8e6-b0368f72fd00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:22.041838 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:22.150575 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 77/162] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:22.151025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:22.151548 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd284e-cd4f-464e-a8e6-b0368f72fd00 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:22.163914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf60fc0-5f84-4687-921a-2f8ef50ba5b3 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:22.241572 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14efb602-a43f-4281-9d81-8be5d68feca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:22.242601 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efb602-a43f-4281-9d81-8be5d68feca4 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:22.242881 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efb602-a43f-4281-9d81-8be5d68feca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:22.243154 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efb602-a43f-4281-9d81-8be5d68feca4 None admin] PATH_INFO: `/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:22.246981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14efb602-a43f-4281-9d81-8be5d68feca4 None admin] RBAC: Authorizing `identity:get_project(project_id=9ea4cc94eaf84d0c8fec375d056242eb)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:22.248445 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14efb602-a43f-4281-9d81-8be5d68feca4 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:22.251950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 86/163] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:21 2022] GET /identity/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb => generated 298 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:22.253453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14efb602-a43f-4281-9d81-8be5d68feca4 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:22.359931 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd284e-cd4f-464e-a8e6-b0368f72fd00 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:22.422888 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3573c-114c-4f0f-a302-742b24d8e43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:22.424036 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3573c-114c-4f0f-a302-742b24d8e43a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:22.425717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3573c-114c-4f0f-a302-742b24d8e43a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:22.426422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3573c-114c-4f0f-a302-742b24d8e43a None admin] PATH_INFO: `/v3/users/9fe3a5506b35415a9373071c0b976e34` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:22.428575 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3573c-114c-4f0f-a302-742b24d8e43a None admin] RBAC: Authorizing `identity:get_user(user_id=9fe3a5506b35415a9373071c0b976e34)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:22.431008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3573c-114c-4f0f-a302-742b24d8e43a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:22.439941 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14efb602-a43f-4281-9d81-8be5d68feca4 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:22.442592 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 78/164] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:22 2022] GET /identity/v3/users/9fe3a5506b35415a9373071c0b976e34 => generated 247 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:22.443820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3573c-114c-4f0f-a302-742b24d8e43a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:22.501429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:22.526282 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a39b2e8-ea6e-48f5-bf55-f8cf11ac8426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:22.526933 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39b2e8-ea6e-48f5-bf55-f8cf11ac8426 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:22.527213 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39b2e8-ea6e-48f5-bf55-f8cf11ac8426 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:22.527487 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39b2e8-ea6e-48f5-bf55-f8cf11ac8426 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:22.551826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a39b2e8-ea6e-48f5-bf55-f8cf11ac8426 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:22.557234 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-0a39b2e8-ea6e-48f5-bf55-f8cf11ac8426 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:17:22.557234 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:22.557234 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:22.557234 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:22.557234 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:22.557234 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:22.557234 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:22.557234 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:22.557234 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:22.557234 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:22.557234 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:22.557234 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:22.557234 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:22.557234 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:22.557234 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:22.557234 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:22.558153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:22.558153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:22.558153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:22.558153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:22.558153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:22.558153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:22.558153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:22.558153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:22.558153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:22.558153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:22.558153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:22.558153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:22.558153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:22.558153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:22.558153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:22.558153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:22.559044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:22.559044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:22.559044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:22.559044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:22.559044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:22.559044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:22.559044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:22.559044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:22.559044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:22.559044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:22.559044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:17:22.559044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:17:22.565077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 87/165] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Sep 1 16:17:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:22.565965 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a39b2e8-ea6e-48f5-bf55-f8cf11ac8426 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:22.617860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:22.664418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3573c-114c-4f0f-a302-742b24d8e43a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:22.729605 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a39b2e8-ea6e-48f5-bf55-f8cf11ac8426 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:22.732748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860ec25-4104-47c0-a1a2-e5dbe0a476b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:22.733629 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860ec25-4104-47c0-a1a2-e5dbe0a476b3 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:22.734091 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860ec25-4104-47c0-a1a2-e5dbe0a476b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:22.734565 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860ec25-4104-47c0-a1a2-e5dbe0a476b3 None admin] PATH_INFO: `/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb/users/ed0ffdea22f44c0f9c9ddb3ba52a22db/roles/d657364ebb9c4110bc1acfe8a9f2f3fd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:22.737738 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860ec25-4104-47c0-a1a2-e5dbe0a476b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ea4cc94eaf84d0c8fec375d056242eb, user_id=ed0ffdea22f44c0f9c9ddb3ba52a22db, role_id=d657364ebb9c4110bc1acfe8a9f2f3fd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:22.742827 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860ec25-4104-47c0-a1a2-e5dbe0a476b3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:22.767447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 79/166] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Sep 1 16:17:22 2022] PUT /identity/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb/users/ed0ffdea22f44c0f9c9ddb3ba52a22db/roles/d657364ebb9c4110bc1acfe8a9f2f3fd => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:17:22.775749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860ec25-4104-47c0-a1a2-e5dbe0a476b3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:22.803028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429580b2-a981-4802-ba49-f65fe295d4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:22.804052 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429580b2-a981-4802-ba49-f65fe295d4f7 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:22.804549 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429580b2-a981-4802-ba49-f65fe295d4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:22.805024 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429580b2-a981-4802-ba49-f65fe295d4f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:22.905698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:22.930999 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860ec25-4104-47c0-a1a2-e5dbe0a476b3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:22.979926 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd51b4-6c50-4bfc-a654-30fa4ae1de1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:22.980474 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd51b4-6c50-4bfc-a654-30fa4ae1de1b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:22.980758 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd51b4-6c50-4bfc-a654-30fa4ae1de1b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:22.981039 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd51b4-6c50-4bfc-a654-30fa4ae1de1b None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:22.981734 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd51b4-6c50-4bfc-a654-30fa4ae1de1b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:22.982042 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd51b4-6c50-4bfc-a654-30fa4ae1de1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:22.983740 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd51b4-6c50-4bfc-a654-30fa4ae1de1b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:23.002523 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429580b2-a981-4802-ba49-f65fe295d4f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:23.004152 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429580b2-a981-4802-ba49-f65fe295d4f7 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:23.006328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 80/167] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:17:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:23.007290 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd51b4-6c50-4bfc-a654-30fa4ae1de1b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:23.059403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:23.152817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 88/168] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:17:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:17:23.154709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb13309-bebd-49b1-9ad1-d7c91b154a4b None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:23.155201 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb13309-bebd-49b1-9ad1-d7c91b154a4b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:23.155791 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb13309-bebd-49b1-9ad1-d7c91b154a4b None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:23.157657 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 89/169] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:23.158623 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb13309-bebd-49b1-9ad1-d7c91b154a4b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:23.167508 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd51b4-6c50-4bfc-a654-30fa4ae1de1b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:23.205455 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7bf88-f4e3-4428-a5fd-db23cf6b77ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:23.206413 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7bf88-f4e3-4428-a5fd-db23cf6b77ef None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:23.206538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7bf88-f4e3-4428-a5fd-db23cf6b77ef None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:23.206805 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7bf88-f4e3-4428-a5fd-db23cf6b77ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:23.218739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c7bf88-f4e3-4428-a5fd-db23cf6b77ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:23.224274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-34c7bf88-f4e3-4428-a5fd-db23cf6b77ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:23.224274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:23.224274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:23.224274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:23.224274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:23.224274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:23.224274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:23.224274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:23.224274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:23.224274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:23.224274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:23.224274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:23.224274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:23.224274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:23.224274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:23.224274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:23.225231 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:23.225231 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:23.225231 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:23.225231 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:23.225231 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:23.225231 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:23.225231 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:23.225231 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:23.225231 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:23.225231 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:23.225231 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:23.225231 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:23.225231 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:23.225231 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:23.225231 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:23.225231 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:23.226535 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:23.226535 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:23.226535 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:23.226535 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:23.226535 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:23.226535 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:23.226535 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:23.226535 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:23.226535 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:23.226535 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:23.226535 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:23.226535 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:17:23.226535 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 81/170] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:23.230242 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7bf88-f4e3-4428-a5fd-db23cf6b77ef None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:23.236009 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:23.281873 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb13309-bebd-49b1-9ad1-d7c91b154a4b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:23.310922 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:23.320652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd15aca-486b-42eb-9063-2f47f6a1d584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:23.321674 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd15aca-486b-42eb-9063-2f47f6a1d584 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:23.322694 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd15aca-486b-42eb-9063-2f47f6a1d584 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:23.323169 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd15aca-486b-42eb-9063-2f47f6a1d584 None admin] PATH_INFO: `/v3/users/3c9bc22bd8da47e98a53e9c6e78236ae` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:23.324579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd15aca-486b-42eb-9063-2f47f6a1d584 None admin] RBAC: Authorizing `identity:get_user(user_id=3c9bc22bd8da47e98a53e9c6e78236ae)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:23.326360 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd15aca-486b-42eb-9063-2f47f6a1d584 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:23.328215 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 90/171] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:23 2022] GET /identity/v3/users/3c9bc22bd8da47e98a53e9c6e78236ae => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:23.329806 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee570a7e-171d-4a0d-a09e-3f850235674d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:23.330346 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee570a7e-171d-4a0d-a09e-3f850235674d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:23.330913 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee570a7e-171d-4a0d-a09e-3f850235674d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:23.350109 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ee570a7e-171d-4a0d-a09e-3f850235674d None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:23.351759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ee570a7e-171d-4a0d-a09e-3f850235674d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:23.364008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7bf88-f4e3-4428-a5fd-db23cf6b77ef None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:23.424091 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 91/172] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:23.425507 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee570a7e-171d-4a0d-a09e-3f850235674d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:23.450934 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ed814-5a37-4f3f-bd88-7cadd149844f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:23.451601 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ed814-5a37-4f3f-bd88-7cadd149844f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:23.451890 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ed814-5a37-4f3f-bd88-7cadd149844f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:23.452170 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ed814-5a37-4f3f-bd88-7cadd149844f None admin] PATH_INFO: `/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:23.453382 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280ed814-5a37-4f3f-bd88-7cadd149844f None admin] RBAC: Authorizing `identity:get_project(project_id=9ea4cc94eaf84d0c8fec375d056242eb)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:23.455174 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280ed814-5a37-4f3f-bd88-7cadd149844f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:23.457155 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 82/173] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:23 2022] GET /identity/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:23.457888 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ed814-5a37-4f3f-bd88-7cadd149844f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:23.579689 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:23.599888 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee570a7e-171d-4a0d-a09e-3f850235674d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:23.626103 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ed814-5a37-4f3f-bd88-7cadd149844f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:23.642920 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:23.657844 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c2b5c-b34e-4010-bbbf-bf1096a9cb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:23.658635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c2b5c-b34e-4010-bbbf-bf1096a9cb34 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:23.658924 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c2b5c-b34e-4010-bbbf-bf1096a9cb34 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:23.659197 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c2b5c-b34e-4010-bbbf-bf1096a9cb34 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:23.659871 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30c2b5c-b34e-4010-bbbf-bf1096a9cb34 None admin] RBAC: Adding query filter params (name=member) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:23.660171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30c2b5c-b34e-4010-bbbf-bf1096a9cb34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:23.661537 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30c2b5c-b34e-4010-bbbf-bf1096a9cb34 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:23.671198 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 92/174] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Sep 1 16:17:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:23.672206 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c2b5c-b34e-4010-bbbf-bf1096a9cb34 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:23.700316 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5cfd3d-d26f-422b-8139-e890cbb809e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:23.700933 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5cfd3d-d26f-422b-8139-e890cbb809e0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:23.701215 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5cfd3d-d26f-422b-8139-e890cbb809e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:23.701490 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5cfd3d-d26f-422b-8139-e890cbb809e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:23.822295 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5cfd3d-d26f-422b-8139-e890cbb809e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:23.823423 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c2b5c-b34e-4010-bbbf-bf1096a9cb34 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:23.823764 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5cfd3d-d26f-422b-8139-e890cbb809e0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:23.862844 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83dcf99-30e3-4051-b4c3-a49c647688e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:23.863375 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83dcf99-30e3-4051-b4c3-a49c647688e4 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:23.863653 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83dcf99-30e3-4051-b4c3-a49c647688e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:23.863923 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83dcf99-30e3-4051-b4c3-a49c647688e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:23.864661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83dcf99-30e3-4051-b4c3-a49c647688e4 None admin] RBAC: Adding query filter params (role.id=ff293acb8551463aafe6970230042b89, scope.project.id=9ea4cc94eaf84d0c8fec375d056242eb, user.id=9fe3a5506b35415a9373071c0b976e34) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:23.865704 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83dcf99-30e3-4051-b4c3-a49c647688e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:23.866443 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83dcf99-30e3-4051-b4c3-a49c647688e4 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:23.873004 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 93/175] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:17:23 2022] GET /identity/v3/role_assignments?user.id=9fe3a5506b35415a9373071c0b976e34&scope.project.id=9ea4cc94eaf84d0c8fec375d056242eb&role.id=ff293acb8551463aafe6970230042b89 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:23.882911 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b002bb-63ca-492d-a592-53627d8906bd None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:23.882911 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b002bb-63ca-492d-a592-53627d8906bd None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:23.882911 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b002bb-63ca-492d-a592-53627d8906bd None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:23.882911 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 94/176] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:23.882911 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b002bb-63ca-492d-a592-53627d8906bd None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:23.917814 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:23.959255 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:23.967561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 83/177] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:17:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:17:23.969275 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5cfd3d-d26f-422b-8139-e890cbb809e0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:24.025942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b002bb-63ca-492d-a592-53627d8906bd None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:24.067880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d3688c-2a07-415c-b662-7a70ccf2233c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:24.068523 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3688c-2a07-415c-b662-7a70ccf2233c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:24.068802 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3688c-2a07-415c-b662-7a70ccf2233c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:24.069073 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3688c-2a07-415c-b662-7a70ccf2233c None admin] PATH_INFO: `/v3/users/3c9bc22bd8da47e98a53e9c6e78236ae` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:24.070277 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d3688c-2a07-415c-b662-7a70ccf2233c None admin] RBAC: Authorizing `identity:get_user(user_id=3c9bc22bd8da47e98a53e9c6e78236ae)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:24.072544 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d3688c-2a07-415c-b662-7a70ccf2233c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:24.073937 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 95/178] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:23 2022] GET /identity/v3/users/3c9bc22bd8da47e98a53e9c6e78236ae => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:24.078306 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3097d1-a1ce-488a-98b9-96b0323da736 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:24.078306 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3097d1-a1ce-488a-98b9-96b0323da736 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:24.078306 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3097d1-a1ce-488a-98b9-96b0323da736 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:24.078306 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 96/179] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:24.078306 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b236d3-73af-4f8d-930a-14138107344c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:24.078306 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b236d3-73af-4f8d-930a-14138107344c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:24.078739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b236d3-73af-4f8d-930a-14138107344c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:24.099028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-09b236d3-73af-4f8d-930a-14138107344c None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:24.100232 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-09b236d3-73af-4f8d-930a-14138107344c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:24.100519 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5cfd3d-d26f-422b-8139-e890cbb809e0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:24.138433 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44210f07-da6e-4e5b-9f43-a6728806cb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:24.138997 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44210f07-da6e-4e5b-9f43-a6728806cb10 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:24.139282 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44210f07-da6e-4e5b-9f43-a6728806cb10 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:24.139555 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44210f07-da6e-4e5b-9f43-a6728806cb10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:24.155828 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44210f07-da6e-4e5b-9f43-a6728806cb10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:24.160794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-44210f07-da6e-4e5b-9f43-a6728806cb10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:17:24.160794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:24.160794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:24.160794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:24.160794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:24.160794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:24.160794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:24.160794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:24.160794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:24.160794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:24.160794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:24.160794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:24.160794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:24.160794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:24.160794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:24.160794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:24.161690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:24.161690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:24.161690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:24.161690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:24.161690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:24.161690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:24.161690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:24.161690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:24.161690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:24.161690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:24.161690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:24.161690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:24.161690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:24.161690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:24.161690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:24.161690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:24.162549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:24.162549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:24.162549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:24.162549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:24.162549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:24.162549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:24.162549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:24.162549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:24.162549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:24.162549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:24.162549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:17:24.162549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:17:24.162549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 84/180] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:17:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:24.167761 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44210f07-da6e-4e5b-9f43-a6728806cb10 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:24.181541 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 97/181] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:24.182517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b236d3-73af-4f8d-930a-14138107344c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:24.200412 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:24.251517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:24.295921 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44210f07-da6e-4e5b-9f43-a6728806cb10 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:24.306861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b236d3-73af-4f8d-930a-14138107344c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:24.347390 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f57030-276a-46fe-825f-eadab3bb4bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:24.348006 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f57030-276a-46fe-825f-eadab3bb4bfe None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:24.348291 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f57030-276a-46fe-825f-eadab3bb4bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:24.348559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f57030-276a-46fe-825f-eadab3bb4bfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:24.349299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32807ab-e101-4565-ad34-21a8ac592f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:24.350095 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32807ab-e101-4565-ad34-21a8ac592f2f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:24.350559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32807ab-e101-4565-ad34-21a8ac592f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:24.350991 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32807ab-e101-4565-ad34-21a8ac592f2f None admin] PATH_INFO: `/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:24.354009 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32807ab-e101-4565-ad34-21a8ac592f2f None admin] RBAC: Authorizing `identity:get_project(project_id=9ea4cc94eaf84d0c8fec375d056242eb)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:24.355638 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32807ab-e101-4565-ad34-21a8ac592f2f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:24.358462 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 85/182] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:24 2022] GET /identity/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:24.359816 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38585e-c3a9-49d3-9969-9b202fb31f31 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:24.360249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38585e-c3a9-49d3-9969-9b202fb31f31 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:24.361166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38585e-c3a9-49d3-9969-9b202fb31f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:24.385456 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5a38585e-c3a9-49d3-9969-9b202fb31f31 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:24.388385 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5a38585e-c3a9-49d3-9969-9b202fb31f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:24.423184 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f57030-276a-46fe-825f-eadab3bb4bfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:24.424744 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f57030-276a-46fe-825f-eadab3bb4bfe None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:24.474613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:24.483882 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 86/183] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:24.485033 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38585e-c3a9-49d3-9969-9b202fb31f31 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:24.528281 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:24.532613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 98/184] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:17:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:17:24.533679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f57030-276a-46fe-825f-eadab3bb4bfe None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:24.612154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38585e-c3a9-49d3-9969-9b202fb31f31 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:24.647833 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46389eb4-1fd5-44e0-a2d4-7a1b0ab73a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:24.648758 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46389eb4-1fd5-44e0-a2d4-7a1b0ab73a10 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:24.649232 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46389eb4-1fd5-44e0-a2d4-7a1b0ab73a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:24.649718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46389eb4-1fd5-44e0-a2d4-7a1b0ab73a10 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:24.650635 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46389eb4-1fd5-44e0-a2d4-7a1b0ab73a10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:24.651129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46389eb4-1fd5-44e0-a2d4-7a1b0ab73a10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:24.652755 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46389eb4-1fd5-44e0-a2d4-7a1b0ab73a10 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:24.661174 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 87/185] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:24.662276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46389eb4-1fd5-44e0-a2d4-7a1b0ab73a10 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:24.672123 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f57030-276a-46fe-825f-eadab3bb4bfe None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:24.708753 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd5375-cd21-4dd7-9d83-1ea98b1dcd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:24.709672 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd5375-cd21-4dd7-9d83-1ea98b1dcd69 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:24.710143 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd5375-cd21-4dd7-9d83-1ea98b1dcd69 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:24.710707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd5375-cd21-4dd7-9d83-1ea98b1dcd69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:24.711639 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd5375-cd21-4dd7-9d83-1ea98b1dcd69 None admin] RBAC: Adding query filter params (role.id=72700efc336a4b4f992b08ff9ea99f0c, scope.project.id=9ea4cc94eaf84d0c8fec375d056242eb, user.id=3c9bc22bd8da47e98a53e9c6e78236ae) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:24.712134 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd5375-cd21-4dd7-9d83-1ea98b1dcd69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:24.713840 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd5375-cd21-4dd7-9d83-1ea98b1dcd69 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:24.720071 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 99/186] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:17:24 2022] GET /identity/v3/role_assignments?user.id=3c9bc22bd8da47e98a53e9c6e78236ae&scope.project.id=9ea4cc94eaf84d0c8fec375d056242eb&role.id=72700efc336a4b4f992b08ff9ea99f0c => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:24.721084 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd5375-cd21-4dd7-9d83-1ea98b1dcd69 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:24.738979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:24.775183 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:24.784613 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46389eb4-1fd5-44e0-a2d4-7a1b0ab73a10 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:24.825549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c01c0-5fd7-46fa-bdf0-d0ea003b9f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:24.826219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c01c0-5fd7-46fa-bdf0-d0ea003b9f58 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:24.826574 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c01c0-5fd7-46fa-bdf0-d0ea003b9f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:24.826908 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c01c0-5fd7-46fa-bdf0-d0ea003b9f58 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:24.837937 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd5375-cd21-4dd7-9d83-1ea98b1dcd69 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:24.841670 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6c01c0-5fd7-46fa-bdf0-d0ea003b9f58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:24.845920 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-4c6c01c0-5fd7-46fa-bdf0-d0ea003b9f58 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:17:24.845920 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:24.845920 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:24.845920 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:24.845920 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:24.845920 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:24.845920 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:24.845920 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:24.845920 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:24.845920 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:24.845920 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:24.845920 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:24.845920 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:24.845920 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:24.845920 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:24.845920 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:24.846948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:24.846948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:24.846948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:24.846948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:24.846948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:24.846948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:24.846948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:24.846948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:24.846948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:24.846948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:24.846948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:24.846948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:24.846948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:24.846948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:24.846948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:24.846948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:24.847791 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:24.847791 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:24.847791 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:24.847791 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:24.847791 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:24.847791 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:24.847791 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:24.847791 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:24.847791 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:24.847791 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:24.847791 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:17:24.847791 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:17:24.847791 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 88/187] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Sep 1 16:17:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:24.848443 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c01c0-5fd7-46fa-bdf0-d0ea003b9f58 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:24.882995 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89993e59-b2ef-41a7-8879-4f1de4e2898c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:24.884031 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89993e59-b2ef-41a7-8879-4f1de4e2898c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:24.885124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89993e59-b2ef-41a7-8879-4f1de4e2898c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:24.885588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89993e59-b2ef-41a7-8879-4f1de4e2898c None admin] PATH_INFO: `/v3/projects/34b5f0c9ed7a4c86a6c36d4534ba0d55` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:24.887339 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89993e59-b2ef-41a7-8879-4f1de4e2898c None admin] RBAC: Authorizing `identity:get_project(project_id=34b5f0c9ed7a4c86a6c36d4534ba0d55)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:24.889075 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89993e59-b2ef-41a7-8879-4f1de4e2898c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:24.891656 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 100/188] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:24 2022] GET /identity/v3/projects/34b5f0c9ed7a4c86a6c36d4534ba0d55 => generated 312 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:24.892865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89993e59-b2ef-41a7-8879-4f1de4e2898c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:24.961218 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c01c0-5fd7-46fa-bdf0-d0ea003b9f58 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:24.994432 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:25.004446 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d1f67-b250-4ab2-9803-95a46a254e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:25.005093 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d1f67-b250-4ab2-9803-95a46a254e46 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:25.005429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d1f67-b250-4ab2-9803-95a46a254e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:25.005708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d1f67-b250-4ab2-9803-95a46a254e46 None admin] PATH_INFO: `/v3/users/8744ed8f950849998c65ca289ccbb34c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:25.020063 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89993e59-b2ef-41a7-8879-4f1de4e2898c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:25.023157 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d1f67-b250-4ab2-9803-95a46a254e46 None admin] RBAC: Authorizing `identity:get_user(user_id=8744ed8f950849998c65ca289ccbb34c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:25.024623 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d1f67-b250-4ab2-9803-95a46a254e46 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:25.027261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 89/189] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:24 2022] GET /identity/v3/users/8744ed8f950849998c65ca289ccbb34c => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:25.028059 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d1f67-b250-4ab2-9803-95a46a254e46 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:25.041089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:25.065500 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ad195-86b6-4147-ad7a-5fdc09ca120a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:25.066142 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ad195-86b6-4147-ad7a-5fdc09ca120a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:25.066466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ad195-86b6-4147-ad7a-5fdc09ca120a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:25.066738 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ad195-86b6-4147-ad7a-5fdc09ca120a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:25.067853 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ad195-86b6-4147-ad7a-5fdc09ca120a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:25.069275 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ad195-86b6-4147-ad7a-5fdc09ca120a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:25.071077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 101/190] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:17:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:25.076065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ad195-86b6-4147-ad7a-5fdc09ca120a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:25.143118 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d1f67-b250-4ab2-9803-95a46a254e46 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:25.180141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe625f4-6eec-497d-8936-a28c166a0220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:25.181041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe625f4-6eec-497d-8936-a28c166a0220 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:25.181866 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe625f4-6eec-497d-8936-a28c166a0220 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:25.182592 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe625f4-6eec-497d-8936-a28c166a0220 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:25.183276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe625f4-6eec-497d-8936-a28c166a0220 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:25.183580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe625f4-6eec-497d-8936-a28c166a0220 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:25.185075 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe625f4-6eec-497d-8936-a28c166a0220 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:25.196366 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ad195-86b6-4147-ad7a-5fdc09ca120a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:25.198679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 90/191] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:17:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:25.199549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe625f4-6eec-497d-8936-a28c166a0220 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:25.230392 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eabb96-d1a2-4759-999b-1b9ec07e9517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:25.230921 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eabb96-d1a2-4759-999b-1b9ec07e9517 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:25.231197 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eabb96-d1a2-4759-999b-1b9ec07e9517 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:25.231466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eabb96-d1a2-4759-999b-1b9ec07e9517 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:25.232190 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eabb96-d1a2-4759-999b-1b9ec07e9517 None admin] RBAC: Adding query filter params (role.id=42db5f955f2644b1bb22b0a68f8cd829, scope.project.id=34b5f0c9ed7a4c86a6c36d4534ba0d55, user.id=3c9bc22bd8da47e98a53e9c6e78236ae) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:25.232497 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eabb96-d1a2-4759-999b-1b9ec07e9517 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:25.233914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eabb96-d1a2-4759-999b-1b9ec07e9517 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:25.240558 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 102/192] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:17:25 2022] GET /identity/v3/role_assignments?user.id=3c9bc22bd8da47e98a53e9c6e78236ae&scope.project.id=34b5f0c9ed7a4c86a6c36d4534ba0d55&role.id=42db5f955f2644b1bb22b0a68f8cd829 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:25.241314 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eabb96-d1a2-4759-999b-1b9ec07e9517 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:25.259796 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:25.283870 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:25.318268 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe625f4-6eec-497d-8936-a28c166a0220 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:25.356035 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eabb96-d1a2-4759-999b-1b9ec07e9517 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:25.362331 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414b4a9-cff5-41ea-b0f1-08ab389dad23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:25.363247 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414b4a9-cff5-41ea-b0f1-08ab389dad23 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:25.364218 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414b4a9-cff5-41ea-b0f1-08ab389dad23 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:25.364680 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414b4a9-cff5-41ea-b0f1-08ab389dad23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:25.395666 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25563556-7f96-4845-a45d-c17c1b40cd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:25.396291 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25563556-7f96-4845-a45d-c17c1b40cd2a None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:25.396568 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25563556-7f96-4845-a45d-c17c1b40cd2a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:25.396836 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25563556-7f96-4845-a45d-c17c1b40cd2a None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:25.397496 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25563556-7f96-4845-a45d-c17c1b40cd2a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:25.398960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25563556-7f96-4845-a45d-c17c1b40cd2a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:25.433553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 103/193] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Sep 1 16:17:25 2022] POST /identity/v3/users => generated 297 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:17:25.434260 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25563556-7f96-4845-a45d-c17c1b40cd2a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:25.474666 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414b4a9-cff5-41ea-b0f1-08ab389dad23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:25.476448 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414b4a9-cff5-41ea-b0f1-08ab389dad23 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:25.515081 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:25.555552 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25563556-7f96-4845-a45d-c17c1b40cd2a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:25.555957 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:25.592104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 91/194] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:17:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:17:25.593508 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c8599-728d-4fcd-9398-9cd72dd5d835 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:25.593792 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c8599-728d-4fcd-9398-9cd72dd5d835 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:25.594061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c8599-728d-4fcd-9398-9cd72dd5d835 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:25.595323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 92/195] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:25.598266 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd67ba0-f341-453b-8f7c-eda76e6eee38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:25.598266 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd67ba0-f341-453b-8f7c-eda76e6eee38 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:25.598266 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd67ba0-f341-453b-8f7c-eda76e6eee38 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:25.598266 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd67ba0-f341-453b-8f7c-eda76e6eee38 None admin] PATH_INFO: `/v3/users/8744ed8f950849998c65ca289ccbb34c` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:25.598592 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c8599-728d-4fcd-9398-9cd72dd5d835 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:25.599868 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd67ba0-f341-453b-8f7c-eda76e6eee38 None admin] RBAC: Authorizing `identity:get_user(user_id=8744ed8f950849998c65ca289ccbb34c)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:25.601446 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd67ba0-f341-453b-8f7c-eda76e6eee38 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:25.603827 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 104/196] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:25 2022] GET /identity/v3/users/8744ed8f950849998c65ca289ccbb34c => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:25.605049 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e72990-a60c-4d04-8e46-0bcebf3b4e82 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:25.605334 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e72990-a60c-4d04-8e46-0bcebf3b4e82 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:25.605605 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e72990-a60c-4d04-8e46-0bcebf3b4e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:25.623635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-75e72990-a60c-4d04-8e46-0bcebf3b4e82 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:25.624815 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-75e72990-a60c-4d04-8e46-0bcebf3b4e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:25.703520 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 105/197] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:25.704166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e72990-a60c-4d04-8e46-0bcebf3b4e82 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:25.740302 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c8599-728d-4fcd-9398-9cd72dd5d835 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:25.782529 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240773e1-fea7-4f15-bcf7-75bc6323e98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:25.783559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240773e1-fea7-4f15-bcf7-75bc6323e98e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:25.784035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240773e1-fea7-4f15-bcf7-75bc6323e98e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:25.784531 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240773e1-fea7-4f15-bcf7-75bc6323e98e None admin] PATH_INFO: `/v3/projects/2375c71530404fc591cb968d121ae21e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:25.785856 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240773e1-fea7-4f15-bcf7-75bc6323e98e None admin] RBAC: Authorizing `identity:get_project(project_id=2375c71530404fc591cb968d121ae21e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:25.786668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:25.788417 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240773e1-fea7-4f15-bcf7-75bc6323e98e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:25.790295 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 93/198] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:25 2022] GET /identity/v3/projects/2375c71530404fc591cb968d121ae21e => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:25.797178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240773e1-fea7-4f15-bcf7-75bc6323e98e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:25.825757 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e72990-a60c-4d04-8e46-0bcebf3b4e82 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:25.854074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:25.870814 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fca1ef-b93a-4a95-884d-8b1663802e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:25.871488 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fca1ef-b93a-4a95-884d-8b1663802e74 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:25.871773 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fca1ef-b93a-4a95-884d-8b1663802e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:25.872047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fca1ef-b93a-4a95-884d-8b1663802e74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:25.934272 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240773e1-fea7-4f15-bcf7-75bc6323e98e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:25.957304 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fca1ef-b93a-4a95-884d-8b1663802e74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:25.959130 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fca1ef-b93a-4a95-884d-8b1663802e74 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:25.974345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb9072-4f77-4bf7-b68d-c74ea60bf394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:25.974955 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb9072-4f77-4bf7-b68d-c74ea60bf394 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:25.975234 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb9072-4f77-4bf7-b68d-c74ea60bf394 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:25.975502 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb9072-4f77-4bf7-b68d-c74ea60bf394 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:25.976236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eb9072-4f77-4bf7-b68d-c74ea60bf394 None admin] RBAC: Adding query filter params (role.id=ff293acb8551463aafe6970230042b89, scope.project.id=2375c71530404fc591cb968d121ae21e, user.id=8744ed8f950849998c65ca289ccbb34c) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:25.976544 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eb9072-4f77-4bf7-b68d-c74ea60bf394 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:25.978031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eb9072-4f77-4bf7-b68d-c74ea60bf394 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:25.983878 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 94/199] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:17:25 2022] GET /identity/v3/role_assignments?user.id=8744ed8f950849998c65ca289ccbb34c&scope.project.id=2375c71530404fc591cb968d121ae21e&role.id=ff293acb8551463aafe6970230042b89 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:26.036373 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:26.050883 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb9072-4f77-4bf7-b68d-c74ea60bf394 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:26.086699 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 106/200] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:17:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:17:26.094522 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fca1ef-b93a-4a95-884d-8b1663802e74 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:26.233319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fca1ef-b93a-4a95-884d-8b1663802e74 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:26.241592 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb9072-4f77-4bf7-b68d-c74ea60bf394 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:26.250064 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:26.270109 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ac886-52e8-466c-9c01-14f5418bfecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:26.270787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ac886-52e8-466c-9c01-14f5418bfecf None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:26.271233 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ac886-52e8-466c-9c01-14f5418bfecf None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:26.271515 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ac886-52e8-466c-9c01-14f5418bfecf None admin] PATH_INFO: `/v3/projects/2375c71530404fc591cb968d121ae21e` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:26.272662 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69ac886-52e8-466c-9c01-14f5418bfecf None admin] RBAC: Authorizing `identity:get_project(project_id=2375c71530404fc591cb968d121ae21e)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:26.274558 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69ac886-52e8-466c-9c01-14f5418bfecf None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:26.277190 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 107/201] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:26 2022] GET /identity/v3/projects/2375c71530404fc591cb968d121ae21e => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:26.283293 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ac886-52e8-466c-9c01-14f5418bfecf None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:26.318703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2165ed-b074-4fdc-8182-8e041daaf027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:26.319812 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2165ed-b074-4fdc-8182-8e041daaf027 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:26.320305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2165ed-b074-4fdc-8182-8e041daaf027 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:26.320778 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2165ed-b074-4fdc-8182-8e041daaf027 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:26.328143 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:26.354561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2165ed-b074-4fdc-8182-8e041daaf027 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:26.360324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-7c2165ed-b074-4fdc-8182-8e041daaf027 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:17:26.360324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:26.360324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:26.360324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:26.360324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:26.360324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:26.360324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:26.360324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:26.360324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:26.360324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:26.360324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:26.360324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:26.360324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:26.360324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:26.360324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:26.360324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:26.361300 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:26.361300 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:26.361300 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:26.361300 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:26.361300 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:26.361300 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:26.361300 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:26.361300 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:26.361300 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:26.361300 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:26.361300 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:26.361300 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:26.361300 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:26.361300 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:26.361300 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:26.361300 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:26.362107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:26.362107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:26.362107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:26.362107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:26.362107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:26.362107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:26.362107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:26.362107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:26.362107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:26.362107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:26.362107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:17:26.362107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:17:26.372268 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 95/202] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:26.378049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2165ed-b074-4fdc-8182-8e041daaf027 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:26.398547 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ac886-52e8-466c-9c01-14f5418bfecf None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:26.437332 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd3e1e-e1ae-4876-b74d-c9c0a0fd90ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:26.437941 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd3e1e-e1ae-4876-b74d-c9c0a0fd90ca None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:26.438240 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd3e1e-e1ae-4876-b74d-c9c0a0fd90ca None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:26.438535 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd3e1e-e1ae-4876-b74d-c9c0a0fd90ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:26.439517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd3e1e-e1ae-4876-b74d-c9c0a0fd90ca None admin] RBAC: Adding query filter params (role.id=72700efc336a4b4f992b08ff9ea99f0c, scope.project.id=2375c71530404fc591cb968d121ae21e, user.id=8744ed8f950849998c65ca289ccbb34c) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:26.439830 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd3e1e-e1ae-4876-b74d-c9c0a0fd90ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:26.441618 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd3e1e-e1ae-4876-b74d-c9c0a0fd90ca None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:26.448642 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 108/203] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:17:26 2022] GET /identity/v3/role_assignments?user.id=8744ed8f950849998c65ca289ccbb34c&scope.project.id=2375c71530404fc591cb968d121ae21e&role.id=72700efc336a4b4f992b08ff9ea99f0c => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:26.586252 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2165ed-b074-4fdc-8182-8e041daaf027 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:26.632354 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57f719-5282-4578-af5c-ab71a1900021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:26.632964 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57f719-5282-4578-af5c-ab71a1900021 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:26.633243 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57f719-5282-4578-af5c-ab71a1900021 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:26.633516 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57f719-5282-4578-af5c-ab71a1900021 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:26.634240 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57f719-5282-4578-af5c-ab71a1900021 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:26.634550 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57f719-5282-4578-af5c-ab71a1900021 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:26.635939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57f719-5282-4578-af5c-ab71a1900021 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:26.645133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 96/204] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Sep 1 16:17:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:26.650556 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd3e1e-e1ae-4876-b74d-c9c0a0fd90ca None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:26.778157 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:26.786046 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd3e1e-e1ae-4876-b74d-c9c0a0fd90ca None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:26.824533 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc471768-ba2e-4d6d-afe3-34c587e70e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:26.825174 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc471768-ba2e-4d6d-afe3-34c587e70e12 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:26.825454 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc471768-ba2e-4d6d-afe3-34c587e70e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:26.825728 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc471768-ba2e-4d6d-afe3-34c587e70e12 None admin] PATH_INFO: `/v3/users/ed0ffdea22f44c0f9c9ddb3ba52a22db` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:26.826875 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc471768-ba2e-4d6d-afe3-34c587e70e12 None admin] RBAC: Authorizing `identity:get_user(user_id=ed0ffdea22f44c0f9c9ddb3ba52a22db)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:26.828326 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc471768-ba2e-4d6d-afe3-34c587e70e12 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:26.829920 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 109/205] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:26 2022] GET /identity/v3/users/ed0ffdea22f44c0f9c9ddb3ba52a22db => generated 289 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:26.835679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57f719-5282-4578-af5c-ab71a1900021 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:26.872999 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:26.949034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57f719-5282-4578-af5c-ab71a1900021 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:26.999905 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa0b04-adb5-4618-bef7-76ca86cae6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:27.000856 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa0b04-adb5-4618-bef7-76ca86cae6c8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:27.001372 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa0b04-adb5-4618-bef7-76ca86cae6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:27.001834 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa0b04-adb5-4618-bef7-76ca86cae6c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:27.095748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa0b04-adb5-4618-bef7-76ca86cae6c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:27.097300 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa0b04-adb5-4618-bef7-76ca86cae6c8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:27.143047 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:27.226716 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 97/206] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:17:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:17:27.235277 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc471768-ba2e-4d6d-afe3-34c587e70e12 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:27.354317 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc471768-ba2e-4d6d-afe3-34c587e70e12 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:27.391903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24b330-bbe9-4b01-9eb0-18bfa3cdacf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:27.392532 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24b330-bbe9-4b01-9eb0-18bfa3cdacf3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:27.392809 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24b330-bbe9-4b01-9eb0-18bfa3cdacf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:27.393131 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24b330-bbe9-4b01-9eb0-18bfa3cdacf3 None admin] PATH_INFO: `/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:27.394222 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d24b330-bbe9-4b01-9eb0-18bfa3cdacf3 None admin] RBAC: Authorizing `identity:get_project(project_id=9ea4cc94eaf84d0c8fec375d056242eb)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:27.395675 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d24b330-bbe9-4b01-9eb0-18bfa3cdacf3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:27.397744 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 110/207] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:27 2022] GET /identity/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:27.404133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa0b04-adb5-4618-bef7-76ca86cae6c8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:27.515146 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa0b04-adb5-4618-bef7-76ca86cae6c8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:27.547173 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:27.563743 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16fbf00-c8bc-43d3-ab4c-caa11594498f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:27.564418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fbf00-c8bc-43d3-ab4c-caa11594498f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:27.564708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fbf00-c8bc-43d3-ab4c-caa11594498f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:27.564987 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fbf00-c8bc-43d3-ab4c-caa11594498f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:27.565934 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16fbf00-c8bc-43d3-ab4c-caa11594498f None admin] RBAC: Adding query filter params (role.id=d657364ebb9c4110bc1acfe8a9f2f3fd, scope.project.id=9ea4cc94eaf84d0c8fec375d056242eb, user.id=ed0ffdea22f44c0f9c9ddb3ba52a22db) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:27.566270 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16fbf00-c8bc-43d3-ab4c-caa11594498f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:27.567724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16fbf00-c8bc-43d3-ab4c-caa11594498f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:27.578688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 98/208] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:17:27 2022] GET /identity/v3/role_assignments?user.id=ed0ffdea22f44c0f9c9ddb3ba52a22db&scope.project.id=9ea4cc94eaf84d0c8fec375d056242eb&role.id=d657364ebb9c4110bc1acfe8a9f2f3fd => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:27.941809 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644f3b4-c303-40a0-8080-e0dd0d8b97f4 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:27.942172 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644f3b4-c303-40a0-8080-e0dd0d8b97f4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:27.942448 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644f3b4-c303-40a0-8080-e0dd0d8b97f4 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:27.943795 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 111/209] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:27.949674 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f206e9d-2a0f-4e50-97a2-53d0d3c003d7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:27.950236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f206e9d-2a0f-4e50-97a2-53d0d3c003d7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:27.950236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f206e9d-2a0f-4e50-97a2-53d0d3c003d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:27.970314 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2f206e9d-2a0f-4e50-97a2-53d0d3c003d7 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:27.971818 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2f206e9d-2a0f-4e50-97a2-53d0d3c003d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:28.061244 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 99/210] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:28.268114 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beba297-c826-4597-bb51-20c24ebe4ad0 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:28.268600 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beba297-c826-4597-bb51-20c24ebe4ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:28.268858 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beba297-c826-4597-bb51-20c24ebe4ad0 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:28.270390 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 112/211] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:28.276153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b74f99-bc43-4c65-a5b5-11374f2dd29a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:28.276436 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b74f99-bc43-4c65-a5b5-11374f2dd29a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:28.276706 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b74f99-bc43-4c65-a5b5-11374f2dd29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:28.286436 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546960c-8627-45d6-8d52-b974830eb98f None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:28.286721 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546960c-8627-45d6-8d52-b974830eb98f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:28.286998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546960c-8627-45d6-8d52-b974830eb98f None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:28.288319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 113/212] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:17:28.294804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec9365-a234-4bea-bf53-2ff4cec48f4f None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:28.295089 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec9365-a234-4bea-bf53-2ff4cec48f4f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:28.295363 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec9365-a234-4bea-bf53-2ff4cec48f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:28.295789 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-05b74f99-bc43-4c65-a5b5-11374f2dd29a None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:28.296965 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-05b74f99-bc43-4c65-a5b5-11374f2dd29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:28.317241 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f5ec9365-a234-4bea-bf53-2ff4cec48f4f None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:28.318986 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f5ec9365-a234-4bea-bf53-2ff4cec48f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:28.355614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:28.368003 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 100/213] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:28.369067 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b74f99-bc43-4c65-a5b5-11374f2dd29a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:28.410547 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 114/214] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:28.427259 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:28.476837 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b74f99-bc43-4c65-a5b5-11374f2dd29a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:28.511164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113de788-7568-47c9-9aab-b0c61ccebe6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:28.511792 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113de788-7568-47c9-9aab-b0c61ccebe6d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:28.512067 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113de788-7568-47c9-9aab-b0c61ccebe6d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:28.512337 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113de788-7568-47c9-9aab-b0c61ccebe6d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:28.522475 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113de788-7568-47c9-9aab-b0c61ccebe6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:28.527641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-113de788-7568-47c9-9aab-b0c61ccebe6d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:28.527641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:28.527641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:28.527641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:28.527641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:28.527641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:28.527641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:28.527641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:28.527641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:28.527641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:28.527641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:28.527641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:28.527641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:28.527641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:28.527641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:28.527641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:28.528543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:28.528543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:28.528543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:28.528543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:28.528543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:28.528543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:28.528543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:28.528543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:28.528543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:28.528543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:28.528543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:28.528543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:28.528543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:28.528543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:28.528543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:28.528543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:28.529371 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:28.529371 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:28.529371 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:28.529371 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:28.529371 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:28.529371 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:28.529371 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:28.529371 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:28.529371 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:28.529371 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:28.529371 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:28.529371 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:17:28.529371 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 101/215] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:28.534670 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec9365-a234-4bea-bf53-2ff4cec48f4f None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:28.638947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec9365-a234-4bea-bf53-2ff4cec48f4f None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:28.649005 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113de788-7568-47c9-9aab-b0c61ccebe6d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:28.677131 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c07426-e2f2-4858-9d3f-5953bacb62da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:28.678259 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c07426-e2f2-4858-9d3f-5953bacb62da None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:28.678751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c07426-e2f2-4858-9d3f-5953bacb62da None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:28.679161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c07426-e2f2-4858-9d3f-5953bacb62da None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:28.679956 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c07426-e2f2-4858-9d3f-5953bacb62da None admin] RBAC: Adding query filter params (name=member) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:28.680414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c07426-e2f2-4858-9d3f-5953bacb62da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:28.682166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c07426-e2f2-4858-9d3f-5953bacb62da None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:28.691317 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 115/216] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Sep 1 16:17:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:28.699054 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c07426-e2f2-4858-9d3f-5953bacb62da None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:28.729273 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:28.761134 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113de788-7568-47c9-9aab-b0c61ccebe6d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:28.771901 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:28.796311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd1cb0-1057-4fbd-bc6d-bc865eb3f554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:28.796872 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd1cb0-1057-4fbd-bc6d-bc865eb3f554 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:28.797155 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd1cb0-1057-4fbd-bc6d-bc865eb3f554 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:28.797432 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd1cb0-1057-4fbd-bc6d-bc865eb3f554 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:28.798574 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd1cb0-1057-4fbd-bc6d-bc865eb3f554 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:28.800118 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd1cb0-1057-4fbd-bc6d-bc865eb3f554 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:28.801190 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c07426-e2f2-4858-9d3f-5953bacb62da None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:28.801684 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 102/217] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:17:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:28.803244 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd1cb0-1057-4fbd-bc6d-bc865eb3f554 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:28.835320 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad7324-bad6-452f-ae27-d0dddc8d53a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:28.835961 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad7324-bad6-452f-ae27-d0dddc8d53a9 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:28.836240 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad7324-bad6-452f-ae27-d0dddc8d53a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:28.836508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad7324-bad6-452f-ae27-d0dddc8d53a9 None admin] PATH_INFO: `/v3/users/da4dd935f69048948da827011e300d74` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:28.849227 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ad7324-bad6-452f-ae27-d0dddc8d53a9 None admin] RBAC: Authorizing `identity:get_user(user_id=da4dd935f69048948da827011e300d74)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:28.850696 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ad7324-bad6-452f-ae27-d0dddc8d53a9 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:28.852282 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 116/218] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:28 2022] GET /identity/v3/users/da4dd935f69048948da827011e300d74 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:28.857756 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da856d-c840-45df-9a1b-756449ec5d56 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:28.857756 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da856d-c840-45df-9a1b-756449ec5d56 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:28.857756 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da856d-c840-45df-9a1b-756449ec5d56 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:28.857756 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 117/219] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:28.858487 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da856d-c840-45df-9a1b-756449ec5d56 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:28.903965 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd1cb0-1057-4fbd-bc6d-bc865eb3f554 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:28.938835 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b0eac-163d-438f-bd09-e133fd6ca7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:28.939457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b0eac-163d-438f-bd09-e133fd6ca7df None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:28.939736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b0eac-163d-438f-bd09-e133fd6ca7df None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:28.940013 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b0eac-163d-438f-bd09-e133fd6ca7df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:28.943255 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:28.950017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b0eac-163d-438f-bd09-e133fd6ca7df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:28.954796 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-6c6b0eac-163d-438f-bd09-e133fd6ca7df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:17:28.954796 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:28.954796 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:28.954796 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:28.954796 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:28.954796 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:28.954796 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:28.954796 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:28.954796 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:28.954796 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:28.954796 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:28.954796 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:28.954796 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:28.954796 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:28.954796 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:28.954796 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:28.955711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:28.955711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:28.955711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:28.955711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:28.955711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:28.955711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:28.955711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:28.955711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:28.955711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:28.955711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:28.955711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:28.955711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:28.955711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:28.955711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:28.955711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:28.955711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:28.956709 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:28.956709 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:28.956709 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:28.956709 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:28.956709 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:28.956709 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:28.956709 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:28.956709 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:28.956709 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:28.956709 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:28.956709 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:17:28.956709 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:17:28.956709 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 103/220] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:17:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:28.956709 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b0eac-163d-438f-bd09-e133fd6ca7df None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:28.962554 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da856d-c840-45df-9a1b-756449ec5d56 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:28.996209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05a6ac-1206-4ed0-acf5-171fcccbc94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:28.996803 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05a6ac-1206-4ed0-acf5-171fcccbc94e None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:28.997077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05a6ac-1206-4ed0-acf5-171fcccbc94e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:28.997345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05a6ac-1206-4ed0-acf5-171fcccbc94e None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:28.997989 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a05a6ac-1206-4ed0-acf5-171fcccbc94e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:28.998184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:28.999487 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a05a6ac-1206-4ed0-acf5-171fcccbc94e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:29.020491 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 118/221] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Sep 1 16:17:28 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:17:29.021661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5359b-14c1-4bf4-8659-59c9e6423ff8 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:29.021941 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5359b-14c1-4bf4-8659-59c9e6423ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:29.022229 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5359b-14c1-4bf4-8659-59c9e6423ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:29.041261 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-11e5359b-14c1-4bf4-8659-59c9e6423ff8 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:29.042771 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-11e5359b-14c1-4bf4-8659-59c9e6423ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:29.090463 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b0eac-163d-438f-bd09-e133fd6ca7df None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:29.122102 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 119/222] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:29.122836 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5359b-14c1-4bf4-8659-59c9e6423ff8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:29.146724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789c8a4-6f5f-4b6f-92c4-e807d1216e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:29.148363 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789c8a4-6f5f-4b6f-92c4-e807d1216e71 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:29.148854 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789c8a4-6f5f-4b6f-92c4-e807d1216e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:29.149605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789c8a4-6f5f-4b6f-92c4-e807d1216e71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:29.220761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:29.242927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1789c8a4-6f5f-4b6f-92c4-e807d1216e71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:29.245606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1789c8a4-6f5f-4b6f-92c4-e807d1216e71 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:29.266648 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5359b-14c1-4bf4-8659-59c9e6423ff8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:29.305944 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:29.314823 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698578d5-5732-4b97-8d6a-84f6063334da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:29.315522 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698578d5-5732-4b97-8d6a-84f6063334da None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:29.315870 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698578d5-5732-4b97-8d6a-84f6063334da None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:29.316209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698578d5-5732-4b97-8d6a-84f6063334da None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:29.317001 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698578d5-5732-4b97-8d6a-84f6063334da None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:29.317370 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698578d5-5732-4b97-8d6a-84f6063334da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:29.318934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698578d5-5732-4b97-8d6a-84f6063334da None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:29.334275 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 120/223] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:29.337097 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698578d5-5732-4b97-8d6a-84f6063334da None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:29.385284 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 104/224] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:17:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:17:29.390927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789c8a4-6f5f-4b6f-92c4-e807d1216e71 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:29.469623 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698578d5-5732-4b97-8d6a-84f6063334da None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:29.506452 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789c8a4-6f5f-4b6f-92c4-e807d1216e71 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:29.509893 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:29.515454 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49c6fba-5a54-40db-a7c7-4961d53161cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:29.516638 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c6fba-5a54-40db-a7c7-4961d53161cd None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:29.516924 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c6fba-5a54-40db-a7c7-4961d53161cd None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:29.517198 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c6fba-5a54-40db-a7c7-4961d53161cd None admin] PATH_INFO: `/v3/users/8744ed8f950849998c65ca289ccbb34c` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:29.518386 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49c6fba-5a54-40db-a7c7-4961d53161cd None admin] RBAC: Authorizing `identity:get_user(user_id=8744ed8f950849998c65ca289ccbb34c)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:29.520313 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49c6fba-5a54-40db-a7c7-4961d53161cd None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:29.526826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 121/225] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:29 2022] GET /identity/v3/users/8744ed8f950849998c65ca289ccbb34c => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:29.526826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49c6fba-5a54-40db-a7c7-4961d53161cd None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:29.556075 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ea788-91f9-4bd0-96f1-258c9ff0198c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:29.556640 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ea788-91f9-4bd0-96f1-258c9ff0198c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:29.556926 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ea788-91f9-4bd0-96f1-258c9ff0198c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:29.557592 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ea788-91f9-4bd0-96f1-258c9ff0198c None admin] PATH_INFO: `/v3/projects/2375c71530404fc591cb968d121ae21e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:29.559019 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ea788-91f9-4bd0-96f1-258c9ff0198c None admin] RBAC: Authorizing `identity:get_project(project_id=2375c71530404fc591cb968d121ae21e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:29.560718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ea788-91f9-4bd0-96f1-258c9ff0198c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:29.563546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 105/226] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:29 2022] GET /identity/v3/projects/2375c71530404fc591cb968d121ae21e => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:29.564781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ea788-91f9-4bd0-96f1-258c9ff0198c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:29.580142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:29.655217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49c6fba-5a54-40db-a7c7-4961d53161cd None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:29.695999 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e8713-358c-4c59-9643-a9ba01e18566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:29.696668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e8713-358c-4c59-9643-a9ba01e18566 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:29.697017 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e8713-358c-4c59-9643-a9ba01e18566 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:29.697273 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e8713-358c-4c59-9643-a9ba01e18566 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:29.699187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ea788-91f9-4bd0-96f1-258c9ff0198c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:29.708865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e8713-358c-4c59-9643-a9ba01e18566 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:29.715996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-8a6e8713-358c-4c59-9643-a9ba01e18566 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:17:29.715996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:29.715996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:29.715996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:29.715996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:29.715996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:29.715996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:29.715996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:29.715996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:29.715996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:29.715996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:29.715996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:29.715996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:29.715996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:29.715996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:29.715996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:29.716941 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:29.716941 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:29.716941 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:29.716941 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:29.716941 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:29.716941 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:29.716941 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:29.716941 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:29.716941 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:29.716941 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:29.716941 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:29.716941 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:29.716941 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:29.716941 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:29.716941 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:29.716941 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:29.717756 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:29.717756 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:29.717756 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:29.717756 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:29.717756 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:29.717756 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:29.717756 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:29.717756 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:29.717756 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:29.717756 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:29.717756 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:17:29.717756 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:17:29.717756 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 122/227] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Sep 1 16:17:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:29.718501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e8713-358c-4c59-9643-a9ba01e18566 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:29.780723 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:29.825260 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e8713-358c-4c59-9643-a9ba01e18566 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:29.835548 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d552875-3d0f-4c5b-95d0-e32da6e53258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:29.836206 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d552875-3d0f-4c5b-95d0-e32da6e53258 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:29.836491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d552875-3d0f-4c5b-95d0-e32da6e53258 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:29.836826 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d552875-3d0f-4c5b-95d0-e32da6e53258 None admin] PATH_INFO: `/v3/projects/2375c71530404fc591cb968d121ae21e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:29.838570 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d552875-3d0f-4c5b-95d0-e32da6e53258 None admin] RBAC: Authorizing `identity:get_project(project_id=2375c71530404fc591cb968d121ae21e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:29.842275 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d552875-3d0f-4c5b-95d0-e32da6e53258 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:29.846212 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 106/228] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:29 2022] GET /identity/v3/projects/2375c71530404fc591cb968d121ae21e => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:29.851002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d552875-3d0f-4c5b-95d0-e32da6e53258 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:29.871009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07685b7-38a1-4926-94c4-a224a29caede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:29.871969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07685b7-38a1-4926-94c4-a224a29caede None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:29.872471 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07685b7-38a1-4926-94c4-a224a29caede None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:29.872934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07685b7-38a1-4926-94c4-a224a29caede None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:29.873926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07685b7-38a1-4926-94c4-a224a29caede None admin] RBAC: Adding query filter params (role.id=42db5f955f2644b1bb22b0a68f8cd829, scope.project.id=2375c71530404fc591cb968d121ae21e, user.id=da4dd935f69048948da827011e300d74) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:29.874479 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07685b7-38a1-4926-94c4-a224a29caede None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:29.876137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07685b7-38a1-4926-94c4-a224a29caede None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:29.882171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 123/229] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:17:29 2022] GET /identity/v3/role_assignments?user.id=da4dd935f69048948da827011e300d74&scope.project.id=2375c71530404fc591cb968d121ae21e&role.id=42db5f955f2644b1bb22b0a68f8cd829 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:29.883319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07685b7-38a1-4926-94c4-a224a29caede None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:30.072959 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:30.086207 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d552875-3d0f-4c5b-95d0-e32da6e53258 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:30.115979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07685b7-38a1-4926-94c4-a224a29caede None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:30.174933 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c7fcaf-4133-4f2d-9f75-abbafdba2d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:30.175572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7fcaf-4133-4f2d-9f75-abbafdba2d61 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:30.175693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:30.175865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7fcaf-4133-4f2d-9f75-abbafdba2d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:30.176579 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7fcaf-4133-4f2d-9f75-abbafdba2d61 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:30.176797 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c7fcaf-4133-4f2d-9f75-abbafdba2d61 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:30.177106 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c7fcaf-4133-4f2d-9f75-abbafdba2d61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:30.178537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c7fcaf-4133-4f2d-9f75-abbafdba2d61 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:30.198538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 107/230] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:17:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:30.204560 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338871c-c067-4e0d-b793-2870375f9c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:30.204690 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338871c-c067-4e0d-b793-2870375f9c5e None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:30.205629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338871c-c067-4e0d-b793-2870375f9c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:30.205629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338871c-c067-4e0d-b793-2870375f9c5e None admin] PATH_INFO: `/v3/projects/2375c71530404fc591cb968d121ae21e/users/8744ed8f950849998c65ca289ccbb34c/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:30.206909 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c7fcaf-4133-4f2d-9f75-abbafdba2d61 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:30.208754 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338871c-c067-4e0d-b793-2870375f9c5e None admin] RBAC: Authorizing `identity:create_grant(project_id=2375c71530404fc591cb968d121ae21e, user_id=8744ed8f950849998c65ca289ccbb34c, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:30.210219 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338871c-c067-4e0d-b793-2870375f9c5e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:30.246522 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 124/231] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Sep 1 16:17:29 2022] PUT /identity/v3/projects/2375c71530404fc591cb968d121ae21e/users/8744ed8f950849998c65ca289ccbb34c/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:17:30.393953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c7fcaf-4133-4f2d-9f75-abbafdba2d61 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:30.448371 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b3ce7-2eff-4d91-b495-a5a7e2a2ed0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:30.449035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b3ce7-2eff-4d91-b495-a5a7e2a2ed0d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:30.449320 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b3ce7-2eff-4d91-b495-a5a7e2a2ed0d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:30.449597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b3ce7-2eff-4d91-b495-a5a7e2a2ed0d None admin] PATH_INFO: `/v3/users/8744ed8f950849998c65ca289ccbb34c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:30.450987 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b3ce7-2eff-4d91-b495-a5a7e2a2ed0d None admin] RBAC: Authorizing `identity:get_user(user_id=8744ed8f950849998c65ca289ccbb34c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:30.452461 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b3ce7-2eff-4d91-b495-a5a7e2a2ed0d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:30.454625 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 108/232] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:30 2022] GET /identity/v3/users/8744ed8f950849998c65ca289ccbb34c => generated 283 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:30.460501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338871c-c067-4e0d-b793-2870375f9c5e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:30.624666 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338871c-c067-4e0d-b793-2870375f9c5e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:30.663187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c11d1c-7a8a-4a55-8667-4ae8d5be2240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:30.663822 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c11d1c-7a8a-4a55-8667-4ae8d5be2240 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:30.664099 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c11d1c-7a8a-4a55-8667-4ae8d5be2240 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:30.664370 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c11d1c-7a8a-4a55-8667-4ae8d5be2240 None admin] PATH_INFO: `/v3/projects/2375c71530404fc591cb968d121ae21e` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:30.666051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c11d1c-7a8a-4a55-8667-4ae8d5be2240 None admin] RBAC: Authorizing `identity:get_project(project_id=2375c71530404fc591cb968d121ae21e)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:30.667777 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c11d1c-7a8a-4a55-8667-4ae8d5be2240 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:30.669404 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 125/233] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:30 2022] GET /identity/v3/projects/2375c71530404fc591cb968d121ae21e => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:30.675217 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b3ce7-2eff-4d91-b495-a5a7e2a2ed0d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:30.679284 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:30.789089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b3ce7-2eff-4d91-b495-a5a7e2a2ed0d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:30.835840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad97e54-22bb-470f-b9ce-dcb25ea4b5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:30.836546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad97e54-22bb-470f-b9ce-dcb25ea4b5ff None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:30.836828 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad97e54-22bb-470f-b9ce-dcb25ea4b5ff None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:30.837099 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad97e54-22bb-470f-b9ce-dcb25ea4b5ff None admin] PATH_INFO: `/v3/projects/2375c71530404fc591cb968d121ae21e/users/8744ed8f950849998c65ca289ccbb34c/roles/72700efc336a4b4f992b08ff9ea99f0c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:30.839894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad97e54-22bb-470f-b9ce-dcb25ea4b5ff None admin] RBAC: Authorizing `identity:create_grant(project_id=2375c71530404fc591cb968d121ae21e, user_id=8744ed8f950849998c65ca289ccbb34c, role_id=72700efc336a4b4f992b08ff9ea99f0c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:30.841400 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad97e54-22bb-470f-b9ce-dcb25ea4b5ff None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:30.876906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 109/234] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Sep 1 16:17:30 2022] PUT /identity/v3/projects/2375c71530404fc591cb968d121ae21e/users/8744ed8f950849998c65ca289ccbb34c/roles/72700efc336a4b4f992b08ff9ea99f0c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:17:31.436280 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c1e6a-bd57-4159-8c01-7aa94cf476a5 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:31.436955 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c1e6a-bd57-4159-8c01-7aa94cf476a5 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:31.437429 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c1e6a-bd57-4159-8c01-7aa94cf476a5 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:31.439262 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 126/235] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:31.444281 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f154c-2fec-4ec7-a232-a79dc2a39479 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:31.444715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f154c-2fec-4ec7-a232-a79dc2a39479 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:31.444883 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f154c-2fec-4ec7-a232-a79dc2a39479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:31.474495 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e34f154c-2fec-4ec7-a232-a79dc2a39479 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:31.475696 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e34f154c-2fec-4ec7-a232-a79dc2a39479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:31.551904 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:31.555667 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38e44f-6499-4e62-87b5-99397de02e31 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:31.557189 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38e44f-6499-4e62-87b5-99397de02e31 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:31.557588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38e44f-6499-4e62-87b5-99397de02e31 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:31.559279 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 127/236] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:31.568635 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 110/237] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:31.570352 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d25c78-a831-4c00-b5d5-6c78584ff87c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:31.570632 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d25c78-a831-4c00-b5d5-6c78584ff87c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:31.570903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d25c78-a831-4c00-b5d5-6c78584ff87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:31.605241 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b7d25c78-a831-4c00-b5d5-6c78584ff87c None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:31.606470 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b7d25c78-a831-4c00-b5d5-6c78584ff87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:31.669683 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:31.691171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 128/238] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:31.912880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f154c-2fec-4ec7-a232-a79dc2a39479 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:32.019542 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f154c-2fec-4ec7-a232-a79dc2a39479 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:32.069633 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618fccc-fb32-4be2-a996-873efc625a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:32.070296 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618fccc-fb32-4be2-a996-873efc625a6a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:32.070579 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618fccc-fb32-4be2-a996-873efc625a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:32.070853 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618fccc-fb32-4be2-a996-873efc625a6a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:32.081852 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e618fccc-fb32-4be2-a996-873efc625a6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:32.086652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-e618fccc-fb32-4be2-a996-873efc625a6a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:17:32.086652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:32.086652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:32.086652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:32.086652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:32.086652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:32.086652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:32.086652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:32.086652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:32.086652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:32.086652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:32.086652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:32.086652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:32.086652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:32.086652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:32.086652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:32.088068 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:32.088068 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:32.088068 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:32.088068 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:32.088068 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:32.088068 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:32.088068 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:32.088068 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:32.088068 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:32.088068 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:32.088068 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:32.088068 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:32.088068 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:32.088068 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:32.088068 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:32.088068 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:32.089282 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:32.089282 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:32.089282 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:32.089282 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:32.089282 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:32.089282 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:32.089282 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:32.089282 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:32.089282 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:32.089282 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:32.089282 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:17:32.089282 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:17:32.089282 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 111/239] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:32.094281 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d25c78-a831-4c00-b5d5-6c78584ff87c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:32.165131 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618fccc-fb32-4be2-a996-873efc625a6a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:32.220990 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d25c78-a831-4c00-b5d5-6c78584ff87c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:32.247185 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:32.260279 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5059754-f678-48be-9213-e1399402fbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:32.260885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5059754-f678-48be-9213-e1399402fbc4 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:32.261161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5059754-f678-48be-9213-e1399402fbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:32.261431 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5059754-f678-48be-9213-e1399402fbc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:32.262104 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5059754-f678-48be-9213-e1399402fbc4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:32.262619 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5059754-f678-48be-9213-e1399402fbc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:32.263992 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5059754-f678-48be-9213-e1399402fbc4 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:32.272722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 129/240] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Sep 1 16:17:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:32.273975 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f45d5-e72a-43fc-8f97-a7b19389bcd6 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:32.274289 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f45d5-e72a-43fc-8f97-a7b19389bcd6 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:32.274564 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f45d5-e72a-43fc-8f97-a7b19389bcd6 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:32.276032 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 130/241] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:32.280916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f45d5-e72a-43fc-8f97-a7b19389bcd6 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:32.309095 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618fccc-fb32-4be2-a996-873efc625a6a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:32.340510 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:32.357993 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2a5b5-bb07-4b07-8c61-5c8c5b22caac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:32.359031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2a5b5-bb07-4b07-8c61-5c8c5b22caac None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:32.359407 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2a5b5-bb07-4b07-8c61-5c8c5b22caac None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:32.359776 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2a5b5-bb07-4b07-8c61-5c8c5b22caac None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:32.361575 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2a5b5-bb07-4b07-8c61-5c8c5b22caac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:32.363469 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2a5b5-bb07-4b07-8c61-5c8c5b22caac None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:32.366274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 112/242] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:17:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:32.367671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37dcdb-6d9d-4735-8421-526c6fd7f283 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:32.368086 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37dcdb-6d9d-4735-8421-526c6fd7f283 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:32.369041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37dcdb-6d9d-4735-8421-526c6fd7f283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:32.395533 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-bb37dcdb-6d9d-4735-8421-526c6fd7f283 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:32.397247 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bb37dcdb-6d9d-4735-8421-526c6fd7f283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:32.414838 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f45d5-e72a-43fc-8f97-a7b19389bcd6 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:32.459008 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e255a50-e7d2-41d5-ba46-6e0e26888ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:32.460191 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e255a50-e7d2-41d5-ba46-6e0e26888ada None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:32.460618 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e255a50-e7d2-41d5-ba46-6e0e26888ada None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:32.461032 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e255a50-e7d2-41d5-ba46-6e0e26888ada None admin] PATH_INFO: `/v3/users/89bcb151185747288af5f6dcee50db27` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:32.479652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e255a50-e7d2-41d5-ba46-6e0e26888ada None admin] RBAC: Authorizing `identity:get_user(user_id=89bcb151185747288af5f6dcee50db27)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:32.481791 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e255a50-e7d2-41d5-ba46-6e0e26888ada None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:32.484317 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 131/243] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:32 2022] GET /identity/v3/users/89bcb151185747288af5f6dcee50db27 => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:32.489001 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 113/244] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:32.489001 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37dcdb-6d9d-4735-8421-526c6fd7f283 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:32.490665 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e255a50-e7d2-41d5-ba46-6e0e26888ada None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:32.546123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:32.595376 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e255a50-e7d2-41d5-ba46-6e0e26888ada None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:32.596831 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:32.597538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37dcdb-6d9d-4735-8421-526c6fd7f283 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:32.632950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b763e-5104-4f51-bf44-7b0fb941fe1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:32.633601 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b763e-5104-4f51-bf44-7b0fb941fe1d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:32.633891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b763e-5104-4f51-bf44-7b0fb941fe1d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:32.634308 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b763e-5104-4f51-bf44-7b0fb941fe1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:32.635909 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd073547-c90c-4cc8-8a94-cfb88d91242d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:32.636760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd073547-c90c-4cc8-8a94-cfb88d91242d None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:32.637234 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd073547-c90c-4cc8-8a94-cfb88d91242d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:32.637710 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd073547-c90c-4cc8-8a94-cfb88d91242d None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:32.638603 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd073547-c90c-4cc8-8a94-cfb88d91242d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:32.640281 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd073547-c90c-4cc8-8a94-cfb88d91242d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:32.679726 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 114/245] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Sep 1 16:17:32 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:17:32.714618 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b763e-5104-4f51-bf44-7b0fb941fe1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:32.716096 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b763e-5104-4f51-bf44-7b0fb941fe1d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:32.824261 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 132/246] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:17:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:17:32.830114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd073547-c90c-4cc8-8a94-cfb88d91242d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:32.908351 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b763e-5104-4f51-bf44-7b0fb941fe1d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:32.917076 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:32.950938 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd073547-c90c-4cc8-8a94-cfb88d91242d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:32.960143 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:32.989097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0087361-0ecc-4c1a-9bf1-aacccf28de85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:32.989842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0087361-0ecc-4c1a-9bf1-aacccf28de85 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:32.990123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0087361-0ecc-4c1a-9bf1-aacccf28de85 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:32.990449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0087361-0ecc-4c1a-9bf1-aacccf28de85 None admin] PATH_INFO: `/v3/projects/2375c71530404fc591cb968d121ae21e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:32.991641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0087361-0ecc-4c1a-9bf1-aacccf28de85 None admin] RBAC: Authorizing `identity:get_project(project_id=2375c71530404fc591cb968d121ae21e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:32.993075 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0087361-0ecc-4c1a-9bf1-aacccf28de85 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:32.995813 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 115/247] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:32 2022] GET /identity/v3/projects/2375c71530404fc591cb968d121ae21e => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:32.996958 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc316f58-1679-49ac-89b1-b3d1866fabbf None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:32.997235 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc316f58-1679-49ac-89b1-b3d1866fabbf None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:32.997508 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc316f58-1679-49ac-89b1-b3d1866fabbf None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:32.998878 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 116/248] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:33.004149 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea5cab-5c32-437f-bda4-369c863a8355 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:33.004431 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea5cab-5c32-437f-bda4-369c863a8355 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:33.004699 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea5cab-5c32-437f-bda4-369c863a8355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:33.026541 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-eaea5cab-5c32-437f-bda4-369c863a8355 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:33.027723 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-eaea5cab-5c32-437f-bda4-369c863a8355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:33.039221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b763e-5104-4f51-bf44-7b0fb941fe1d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:33.093501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b9016-cfd7-4395-b721-898e6ce08aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:33.094059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b9016-cfd7-4395-b721-898e6ce08aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:33.094368 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b9016-cfd7-4395-b721-898e6ce08aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:33.094643 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b9016-cfd7-4395-b721-898e6ce08aa1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:33.108086 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b9016-cfd7-4395-b721-898e6ce08aa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:33.113727 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-8a1b9016-cfd7-4395-b721-898e6ce08aa1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:33.113727 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:33.113727 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:33.113727 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:33.113727 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:33.113727 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:33.113727 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:33.113727 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:33.113727 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:33.113727 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:33.113727 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:33.113727 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:33.113727 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:33.113727 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:33.113727 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:33.113727 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:33.114854 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:33.114854 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:33.114854 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:33.114854 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:33.114854 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:33.114854 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:33.114854 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:33.114854 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:33.114854 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:33.114854 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:33.114854 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:33.114854 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:33.114854 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:33.114854 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:33.114854 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:33.114854 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:33.115647 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:33.115647 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:33.115647 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:33.115647 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:33.115647 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:33.115647 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:33.115647 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:33.115647 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:33.115647 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:33.115647 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:33.115647 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:33.115647 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:17:33.117559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 117/249] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:17:33.119010 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea5cab-5c32-437f-bda4-369c863a8355 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:33.119219 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 133/250] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:17:33.125791 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b9016-cfd7-4395-b721-898e6ce08aa1 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:33.208588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:33.241584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea5cab-5c32-437f-bda4-369c863a8355 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:33.250066 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:33.260397 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b9016-cfd7-4395-b721-898e6ce08aa1 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:33.278910 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af480788-8ca4-48d0-b4bd-2a070d09c649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:33.279601 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af480788-8ca4-48d0-b4bd-2a070d09c649 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:33.279944 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af480788-8ca4-48d0-b4bd-2a070d09c649 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:33.280278 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af480788-8ca4-48d0-b4bd-2a070d09c649 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:33.281077 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af480788-8ca4-48d0-b4bd-2a070d09c649 None admin] RBAC: Adding query filter params (role.id=d657364ebb9c4110bc1acfe8a9f2f3fd, scope.project.id=2375c71530404fc591cb968d121ae21e, user.id=89bcb151185747288af5f6dcee50db27) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:33.282363 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af480788-8ca4-48d0-b4bd-2a070d09c649 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:33.283874 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af480788-8ca4-48d0-b4bd-2a070d09c649 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:33.291283 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 118/251] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:17:33 2022] GET /identity/v3/role_assignments?user.id=89bcb151185747288af5f6dcee50db27&scope.project.id=2375c71530404fc591cb968d121ae21e&role.id=d657364ebb9c4110bc1acfe8a9f2f3fd => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:33.324373 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0250df-892c-4676-917f-176120100b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:33.325001 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0250df-892c-4676-917f-176120100b05 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:33.325282 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0250df-892c-4676-917f-176120100b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:33.325555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0250df-892c-4676-917f-176120100b05 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:33.326390 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0250df-892c-4676-917f-176120100b05 None admin] RBAC: Adding query filter params (name=member) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:33.326696 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0250df-892c-4676-917f-176120100b05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:33.328084 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0250df-892c-4676-917f-176120100b05 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:33.345904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 134/252] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Sep 1 16:17:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:33.351853 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af480788-8ca4-48d0-b4bd-2a070d09c649 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:33.454028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15227e4b-6c87-4220-98eb-bd3735421cb2 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:33.454357 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15227e4b-6c87-4220-98eb-bd3735421cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:33.454637 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15227e4b-6c87-4220-98eb-bd3735421cb2 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:33.456354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 135/253] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:33.467483 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40bcb1c-3d46-4aa3-9389-ea6b77069792 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:33.467807 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40bcb1c-3d46-4aa3-9389-ea6b77069792 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:33.469984 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40bcb1c-3d46-4aa3-9389-ea6b77069792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:33.492487 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af480788-8ca4-48d0-b4bd-2a070d09c649 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:33.521610 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f40bcb1c-3d46-4aa3-9389-ea6b77069792 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:33.523045 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f40bcb1c-3d46-4aa3-9389-ea6b77069792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:33.530756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b6640-892e-4cd5-b057-76c0db965f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:33.532327 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b6640-892e-4cd5-b057-76c0db965f9a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:33.532612 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b6640-892e-4cd5-b057-76c0db965f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:33.532887 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b6640-892e-4cd5-b057-76c0db965f9a None admin] PATH_INFO: `/v3/users/da4dd935f69048948da827011e300d74` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:33.534048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b6640-892e-4cd5-b057-76c0db965f9a None admin] RBAC: Authorizing `identity:get_user(user_id=da4dd935f69048948da827011e300d74)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:33.535531 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b6640-892e-4cd5-b057-76c0db965f9a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:33.537718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 119/254] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:33 2022] GET /identity/v3/users/da4dd935f69048948da827011e300d74 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:33.544420 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b6640-892e-4cd5-b057-76c0db965f9a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:33.589575 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:33.658049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b6640-892e-4cd5-b057-76c0db965f9a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:33.668846 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 136/255] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:17:33.686985 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40bcb1c-3d46-4aa3-9389-ea6b77069792 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:33.694619 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3760373-c9ce-4562-ae0a-9e1bf612fc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:33.695262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3760373-c9ce-4562-ae0a-9e1bf612fc6e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:33.695537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3760373-c9ce-4562-ae0a-9e1bf612fc6e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:33.695813 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3760373-c9ce-4562-ae0a-9e1bf612fc6e None admin] PATH_INFO: `/v3/projects/2375c71530404fc591cb968d121ae21e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:33.696979 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3760373-c9ce-4562-ae0a-9e1bf612fc6e None admin] RBAC: Authorizing `identity:get_project(project_id=2375c71530404fc591cb968d121ae21e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:33.698472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3760373-c9ce-4562-ae0a-9e1bf612fc6e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:33.704596 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 120/256] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:33 2022] GET /identity/v3/projects/2375c71530404fc591cb968d121ae21e => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:33.707076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3760373-c9ce-4562-ae0a-9e1bf612fc6e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:33.805815 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:33.821338 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3760373-c9ce-4562-ae0a-9e1bf612fc6e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:33.828858 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:33.859437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1601cce4-8212-4844-ab86-3eaf020df96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:33.860561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1601cce4-8212-4844-ab86-3eaf020df96a None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:33.860843 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1601cce4-8212-4844-ab86-3eaf020df96a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:33.861111 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1601cce4-8212-4844-ab86-3eaf020df96a None admin] PATH_INFO: `/v3/projects/2375c71530404fc591cb968d121ae21e/users/da4dd935f69048948da827011e300d74/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:33.864759 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1601cce4-8212-4844-ab86-3eaf020df96a None admin] RBAC: Authorizing `identity:create_grant(project_id=2375c71530404fc591cb968d121ae21e, user_id=da4dd935f69048948da827011e300d74, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:33.866231 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1601cce4-8212-4844-ab86-3eaf020df96a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:33.882338 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 121/257] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Sep 1 16:17:33 2022] PUT /identity/v3/projects/2375c71530404fc591cb968d121ae21e/users/da4dd935f69048948da827011e300d74/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:17:33.913214 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40bcb1c-3d46-4aa3-9389-ea6b77069792 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:33.971941 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514d8d9-2117-46af-bb61-34ddcff76b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:33.972602 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514d8d9-2117-46af-bb61-34ddcff76b82 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:33.972885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514d8d9-2117-46af-bb61-34ddcff76b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:33.973158 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514d8d9-2117-46af-bb61-34ddcff76b82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:33.987234 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514d8d9-2117-46af-bb61-34ddcff76b82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:33.992248 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-1514d8d9-2117-46af-bb61-34ddcff76b82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:17:33.992248 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:33.992248 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:33.992248 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:33.992248 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:33.992248 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:33.992248 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:33.992248 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:33.992248 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:33.992248 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:33.992248 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:33.992248 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:33.992248 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:33.992248 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:33.992248 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:33.992248 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:33.993925 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:33.993925 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:33.993925 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:33.993925 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:33.993925 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:33.993925 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:33.993925 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:33.993925 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:33.993925 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:33.993925 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:33.993925 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:33.993925 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:33.993925 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:33.993925 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:33.993925 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:33.993925 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:33.994799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:33.994799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:33.994799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:33.994799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:33.994799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:33.994799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:33.994799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:33.994799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:33.994799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:33.994799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:33.994799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:17:33.994799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:17:33.994799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 137/258] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:17:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:17:34.000251 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1601cce4-8212-4844-ab86-3eaf020df96a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:34.098246 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514d8d9-2117-46af-bb61-34ddcff76b82 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:34.139808 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1601cce4-8212-4844-ab86-3eaf020df96a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:34.182063 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8842331-aff1-4a6f-a3e4-d16ec811ae4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:34.182644 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8842331-aff1-4a6f-a3e4-d16ec811ae4b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:34.182928 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8842331-aff1-4a6f-a3e4-d16ec811ae4b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:34.183202 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8842331-aff1-4a6f-a3e4-d16ec811ae4b None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:34.183877 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8842331-aff1-4a6f-a3e4-d16ec811ae4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:34.184180 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8842331-aff1-4a6f-a3e4-d16ec811ae4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:34.185568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8842331-aff1-4a6f-a3e4-d16ec811ae4b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:34.203666 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 122/259] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:34.210519 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8842331-aff1-4a6f-a3e4-d16ec811ae4b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:34.266555 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:34.304596 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514d8d9-2117-46af-bb61-34ddcff76b82 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:34.339326 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:34.374337 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407f5e4-e9b5-41cd-a125-49e89a2a2501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:34.374972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407f5e4-e9b5-41cd-a125-49e89a2a2501 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:34.375250 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407f5e4-e9b5-41cd-a125-49e89a2a2501 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:34.375522 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407f5e4-e9b5-41cd-a125-49e89a2a2501 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:34.400285 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2407f5e4-e9b5-41cd-a125-49e89a2a2501 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:34.413115 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8842331-aff1-4a6f-a3e4-d16ec811ae4b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:34.421914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-2407f5e4-e9b5-41cd-a125-49e89a2a2501 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:17:34.421914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:34.421914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:34.421914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:34.421914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:34.421914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:34.421914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:34.421914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:34.421914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:34.421914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:34.421914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:34.421914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:34.421914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:34.421914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:34.421914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:34.421914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:34.422947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:34.422947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:34.422947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:34.422947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:34.422947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:34.422947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:34.422947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:34.422947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:34.422947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:34.422947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:34.422947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:34.422947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:34.422947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:34.422947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:34.422947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:34.422947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:34.423780 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:34.423780 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:34.423780 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:34.423780 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:34.423780 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:34.423780 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:34.423780 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:34.423780 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:34.423780 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:34.423780 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:34.423780 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:17:34.423780 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:17:34.425638 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 138/260] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Sep 1 16:17:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:17:34.430834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407f5e4-e9b5-41cd-a125-49e89a2a2501 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:34.480275 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5184357d-fbb4-4896-a905-e9edda11c361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:34.480886 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5184357d-fbb4-4896-a905-e9edda11c361 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:34.481171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5184357d-fbb4-4896-a905-e9edda11c361 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:34.481448 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5184357d-fbb4-4896-a905-e9edda11c361 None admin] PATH_INFO: `/v3/users/8744ed8f950849998c65ca289ccbb34c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:34.482781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5184357d-fbb4-4896-a905-e9edda11c361 None admin] RBAC: Authorizing `identity:get_user(user_id=8744ed8f950849998c65ca289ccbb34c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:34.484365 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5184357d-fbb4-4896-a905-e9edda11c361 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:34.486110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 123/261] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:34 2022] GET /identity/v3/users/8744ed8f950849998c65ca289ccbb34c => generated 283 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:34.498820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5184357d-fbb4-4896-a905-e9edda11c361 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:34.636077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407f5e4-e9b5-41cd-a125-49e89a2a2501 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:34.660916 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:34.670831 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5184357d-fbb4-4896-a905-e9edda11c361 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:34.675635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab94a83a-18b4-41d1-9cbf-14c0b66a791b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:34.676252 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94a83a-18b4-41d1-9cbf-14c0b66a791b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:34.676593 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94a83a-18b4-41d1-9cbf-14c0b66a791b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:34.676871 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94a83a-18b4-41d1-9cbf-14c0b66a791b None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:34.677541 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab94a83a-18b4-41d1-9cbf-14c0b66a791b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:34.677841 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab94a83a-18b4-41d1-9cbf-14c0b66a791b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:34.679244 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab94a83a-18b4-41d1-9cbf-14c0b66a791b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:34.688567 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 139/262] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:17:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:34.695162 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab94a83a-18b4-41d1-9cbf-14c0b66a791b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:34.737357 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d250c-569d-4823-9a85-df9da6606241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:34.737995 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d250c-569d-4823-9a85-df9da6606241 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:34.738368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d250c-569d-4823-9a85-df9da6606241 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:34.738650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d250c-569d-4823-9a85-df9da6606241 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:34.754581 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:34.830402 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab94a83a-18b4-41d1-9cbf-14c0b66a791b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:34.898216 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d250c-569d-4823-9a85-df9da6606241 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:34.900678 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d250c-569d-4823-9a85-df9da6606241 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:34.917878 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7175b3c-1086-4607-881d-ec82eb1a6e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:34.918566 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7175b3c-1086-4607-881d-ec82eb1a6e5f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:34.918854 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7175b3c-1086-4607-881d-ec82eb1a6e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:34.919131 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7175b3c-1086-4607-881d-ec82eb1a6e5f None admin] PATH_INFO: `/v3/users/8744ed8f950849998c65ca289ccbb34c` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:34.920828 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7175b3c-1086-4607-881d-ec82eb1a6e5f None admin] RBAC: Authorizing `identity:get_user(user_id=8744ed8f950849998c65ca289ccbb34c)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:34.922458 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7175b3c-1086-4607-881d-ec82eb1a6e5f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:34.924171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 140/263] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:34 2022] GET /identity/v3/users/8744ed8f950849998c65ca289ccbb34c => generated 283 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:34.946257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7175b3c-1086-4607-881d-ec82eb1a6e5f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:35.147252 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:35.166346 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 124/264] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:17:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 668 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 16:17:35.171978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d250c-569d-4823-9a85-df9da6606241 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:35.204991 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7175b3c-1086-4607-881d-ec82eb1a6e5f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:35.211233 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:35.308017 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c166b-8dac-4e1d-aa79-13c8fe85c754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:35.308553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c166b-8dac-4e1d-aa79-13c8fe85c754 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:35.308838 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c166b-8dac-4e1d-aa79-13c8fe85c754 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:35.309112 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c166b-8dac-4e1d-aa79-13c8fe85c754 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:35.416933 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d250c-569d-4823-9a85-df9da6606241 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:35.418775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c166b-8dac-4e1d-aa79-13c8fe85c754 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:35.420445 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c166b-8dac-4e1d-aa79-13c8fe85c754 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:35.502673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539b2fa-b6b0-418a-9dbe-4d43988a6e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:35.503391 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539b2fa-b6b0-418a-9dbe-4d43988a6e72 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:35.503622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539b2fa-b6b0-418a-9dbe-4d43988a6e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:35.503903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539b2fa-b6b0-418a-9dbe-4d43988a6e72 None admin] PATH_INFO: `/v3/projects/2375c71530404fc591cb968d121ae21e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:35.505411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7539b2fa-b6b0-418a-9dbe-4d43988a6e72 None admin] RBAC: Authorizing `identity:get_project(project_id=2375c71530404fc591cb968d121ae21e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:35.506730 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7539b2fa-b6b0-418a-9dbe-4d43988a6e72 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:35.508867 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 125/265] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:35 2022] GET /identity/v3/projects/2375c71530404fc591cb968d121ae21e => generated 302 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:35.515304 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539b2fa-b6b0-418a-9dbe-4d43988a6e72 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:35.548009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 141/266] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:17:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 16:17:35.555309 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c166b-8dac-4e1d-aa79-13c8fe85c754 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:35.559930 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:35.630731 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:35.710983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539b2fa-b6b0-418a-9dbe-4d43988a6e72 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:35.720909 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c166b-8dac-4e1d-aa79-13c8fe85c754 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:35.780305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d29954-c4b8-4763-b564-d9763cbcc704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:35.780920 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d29954-c4b8-4763-b564-d9763cbcc704 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:35.781286 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d29954-c4b8-4763-b564-d9763cbcc704 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:35.781568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d29954-c4b8-4763-b564-d9763cbcc704 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:35.782343 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d29954-c4b8-4763-b564-d9763cbcc704 None admin] RBAC: Adding query filter params (role.id=ff293acb8551463aafe6970230042b89, scope.project.id=2375c71530404fc591cb968d121ae21e, user.id=8744ed8f950849998c65ca289ccbb34c) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:35.782644 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d29954-c4b8-4763-b564-d9763cbcc704 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:35.784199 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d29954-c4b8-4763-b564-d9763cbcc704 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:35.790822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 126/267] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:17:35 2022] GET /identity/v3/role_assignments?user.id=8744ed8f950849998c65ca289ccbb34c&scope.project.id=2375c71530404fc591cb968d121ae21e&role.id=ff293acb8551463aafe6970230042b89 => generated 608 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:35.792282 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0f4ce-ac59-43ac-aeca-e79505bacc4d None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:35.792571 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0f4ce-ac59-43ac-aeca-e79505bacc4d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:35.792847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0f4ce-ac59-43ac-aeca-e79505bacc4d None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:35.794061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 127/268] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:35.799694 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a0c9b-1d02-4b98-8a45-0ebb043bae40 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:35.799972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a0c9b-1d02-4b98-8a45-0ebb043bae40 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:35.800366 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a0c9b-1d02-4b98-8a45-0ebb043bae40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:35.819216 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5eb2ba-383c-41c9-9dea-4f9e494ad666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:35.819795 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5eb2ba-383c-41c9-9dea-4f9e494ad666 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:35.820076 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5eb2ba-383c-41c9-9dea-4f9e494ad666 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:35.820349 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5eb2ba-383c-41c9-9dea-4f9e494ad666 None admin] PATH_INFO: `/v3/projects/2375c71530404fc591cb968d121ae21e` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:35.822296 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5eb2ba-383c-41c9-9dea-4f9e494ad666 None admin] RBAC: Authorizing `identity:get_project(project_id=2375c71530404fc591cb968d121ae21e)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:35.823810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5eb2ba-383c-41c9-9dea-4f9e494ad666 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:35.825861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 142/269] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:35 2022] GET /identity/v3/projects/2375c71530404fc591cb968d121ae21e => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:35.826650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4a9a0c9b-1d02-4b98-8a45-0ebb043bae40 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:35.827862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4a9a0c9b-1d02-4b98-8a45-0ebb043bae40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:35.837146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5eb2ba-383c-41c9-9dea-4f9e494ad666 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:35.972527 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 128/270] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:17:36.007985 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:36.009780 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f51610-d6af-4518-8781-617a6f821011 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:36.010073 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f51610-d6af-4518-8781-617a6f821011 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:36.010379 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f51610-d6af-4518-8781-617a6f821011 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:36.012091 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 129/271] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:36.019467 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a6683-b715-4524-821c-54af8210f0fd None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:36.019747 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a6683-b715-4524-821c-54af8210f0fd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:36.020021 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a6683-b715-4524-821c-54af8210f0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:36.042550 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-480a6683-b715-4524-821c-54af8210f0fd None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:36.044209 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-480a6683-b715-4524-821c-54af8210f0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:36.070239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5eb2ba-383c-41c9-9dea-4f9e494ad666 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:36.147379 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc30d4-a697-4883-aa2e-7874476040c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:36.148085 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:36.149294 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc30d4-a697-4883-aa2e-7874476040c4 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:36.151485 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc30d4-a697-4883-aa2e-7874476040c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:36.151981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc30d4-a697-4883-aa2e-7874476040c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:36.153247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc30d4-a697-4883-aa2e-7874476040c4 None admin] RBAC: Adding query filter params (role.id=72700efc336a4b4f992b08ff9ea99f0c, scope.project.id=2375c71530404fc591cb968d121ae21e, user.id=8744ed8f950849998c65ca289ccbb34c) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:36.153938 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc30d4-a697-4883-aa2e-7874476040c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:36.157134 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc30d4-a697-4883-aa2e-7874476040c4 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:36.171427 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 143/272] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:17:35 2022] GET /identity/v3/role_assignments?user.id=8744ed8f950849998c65ca289ccbb34c&scope.project.id=2375c71530404fc591cb968d121ae21e&role.id=72700efc336a4b4f992b08ff9ea99f0c => generated 608 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:36.175876 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 130/273] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:36.211673 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d198b6b-a154-4f90-8c87-031a969c9659 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:36.212859 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d198b6b-a154-4f90-8c87-031a969c9659 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:36.213384 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d198b6b-a154-4f90-8c87-031a969c9659 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:36.215063 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 144/274] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:36.220614 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed5d7c-28d2-46f5-bc03-1766d9a0f490 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:36.220903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed5d7c-28d2-46f5-bc03-1766d9a0f490 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:36.221182 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed5d7c-28d2-46f5-bc03-1766d9a0f490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:36.240838 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b1ed5d7c-28d2-46f5-bc03-1766d9a0f490 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:36.242035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b1ed5d7c-28d2-46f5-bc03-1766d9a0f490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:36.308076 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d198b6b-a154-4f90-8c87-031a969c9659 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:36.360133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 131/275] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:36.457482 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d198b6b-a154-4f90-8c87-031a969c9659 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:36.498967 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d777273-e8d8-4abe-b267-6f1d05e9074f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:36.499958 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d777273-e8d8-4abe-b267-6f1d05e9074f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:36.500435 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d777273-e8d8-4abe-b267-6f1d05e9074f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:36.500966 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d777273-e8d8-4abe-b267-6f1d05e9074f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:36.503881 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:36.517644 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d777273-e8d8-4abe-b267-6f1d05e9074f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:36.522857 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-6d777273-e8d8-4abe-b267-6f1d05e9074f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:36.522857 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:36.522857 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:36.522857 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:36.522857 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:36.522857 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:36.522857 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:36.522857 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:36.522857 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:36.522857 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:36.522857 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:36.522857 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:36.522857 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:36.522857 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:36.522857 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:36.522857 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:36.523747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:36.523747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:36.523747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:36.523747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:36.523747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:36.523747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:36.523747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:36.523747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:36.523747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:36.523747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:36.523747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:36.523747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:36.523747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:36.523747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:36.523747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:36.523747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:36.524555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:36.524555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:36.524555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:36.524555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:36.524555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:36.524555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:36.524555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:36.524555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:36.524555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:36.524555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:36.524555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:36.524555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:17:36.527523 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 145/276] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:36.533435 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed5d7c-28d2-46f5-bc03-1766d9a0f490 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:36.586959 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d777273-e8d8-4abe-b267-6f1d05e9074f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:36.653649 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed5d7c-28d2-46f5-bc03-1766d9a0f490 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:36.680894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:36.702629 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec40e63-df9e-4cab-a114-579c27e71b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:36.703263 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec40e63-df9e-4cab-a114-579c27e71b04 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:36.703547 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec40e63-df9e-4cab-a114-579c27e71b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:36.703825 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec40e63-df9e-4cab-a114-579c27e71b04 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:36.704498 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec40e63-df9e-4cab-a114-579c27e71b04 None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:36.704804 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec40e63-df9e-4cab-a114-579c27e71b04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:36.706219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec40e63-df9e-4cab-a114-579c27e71b04 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:36.715783 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 132/277] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Sep 1 16:17:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:36.716639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec40e63-df9e-4cab-a114-579c27e71b04 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:36.723742 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d777273-e8d8-4abe-b267-6f1d05e9074f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:36.761876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae13a4-604a-49f1-9ba4-35ddc5b98ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:36.762535 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae13a4-604a-49f1-9ba4-35ddc5b98ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:36.762841 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae13a4-604a-49f1-9ba4-35ddc5b98ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:36.763115 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae13a4-604a-49f1-9ba4-35ddc5b98ae7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:36.764222 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae13a4-604a-49f1-9ba4-35ddc5b98ae7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:36.765652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae13a4-604a-49f1-9ba4-35ddc5b98ae7 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:36.767222 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 146/278] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:17:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:36.768030 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae13a4-604a-49f1-9ba4-35ddc5b98ae7 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:36.824797 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec40e63-df9e-4cab-a114-579c27e71b04 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:36.847274 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:36.862751 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d92be0-52d8-4c1d-8617-797e9c994574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:36.863724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d92be0-52d8-4c1d-8617-797e9c994574 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:36.864167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d92be0-52d8-4c1d-8617-797e9c994574 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:36.864589 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d92be0-52d8-4c1d-8617-797e9c994574 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:36.878166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d92be0-52d8-4c1d-8617-797e9c994574 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:36.884693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-37d92be0-52d8-4c1d-8617-797e9c994574 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:17:36.884693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:36.884693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:36.884693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:36.884693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:36.884693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:36.884693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:36.884693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:36.884693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:36.884693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:36.884693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:36.884693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:36.884693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:36.884693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:36.884693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:36.884693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:36.885865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:36.885865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:36.885865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:36.885865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:36.885865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:36.885865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:36.885865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:36.885865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:36.885865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:36.885865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:36.885865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:36.885865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:36.885865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:36.885865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:36.885865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:36.885865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:36.886954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:36.886954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:36.886954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:36.886954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:36.886954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:36.886954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:36.886954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:36.886954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:36.886954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:36.886954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:36.886954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:17:36.886954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:17:36.886954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 133/279] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:36.887903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae13a4-604a-49f1-9ba4-35ddc5b98ae7 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:36.887976 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c667f-bf60-4129-800e-361e10d8902d None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:36.887976 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c667f-bf60-4129-800e-361e10d8902d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:36.887976 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c667f-bf60-4129-800e-361e10d8902d None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:36.889122 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 134/280] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:36.890362 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c667f-bf60-4129-800e-361e10d8902d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:36.924234 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b1ea5-2270-4425-9dac-323dede34842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:36.924942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b1ea5-2270-4425-9dac-323dede34842 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:36.925224 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b1ea5-2270-4425-9dac-323dede34842 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:36.925500 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b1ea5-2270-4425-9dac-323dede34842 None admin] PATH_INFO: `/v3/users/e179ef144e1147faad8f3e61bc0a8e88` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:36.933499 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:36.940100 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b1ea5-2270-4425-9dac-323dede34842 None admin] RBAC: Authorizing `identity:get_user(user_id=e179ef144e1147faad8f3e61bc0a8e88)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:36.941575 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b1ea5-2270-4425-9dac-323dede34842 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:36.943229 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 147/281] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:36 2022] GET /identity/v3/users/e179ef144e1147faad8f3e61bc0a8e88 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:36.948193 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b1ea5-2270-4425-9dac-323dede34842 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:36.992234 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c667f-bf60-4129-800e-361e10d8902d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:37.025320 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc06f7-5013-4672-82f7-b872b8d66eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:37.025918 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc06f7-5013-4672-82f7-b872b8d66eb8 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:37.026286 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc06f7-5013-4672-82f7-b872b8d66eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:37.026492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc06f7-5013-4672-82f7-b872b8d66eb8 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:37.027133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc06f7-5013-4672-82f7-b872b8d66eb8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:37.028572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc06f7-5013-4672-82f7-b872b8d66eb8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:37.044748 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b1ea5-2270-4425-9dac-323dede34842 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:37.050741 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 135/282] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Sep 1 16:17:36 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:17:37.051958 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8a581-9928-499e-887b-4af4cfb119a6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:37.052244 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8a581-9928-499e-887b-4af4cfb119a6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:37.052518 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8a581-9928-499e-887b-4af4cfb119a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:37.072269 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ccc8a581-9928-499e-887b-4af4cfb119a6 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:37.073506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ccc8a581-9928-499e-887b-4af4cfb119a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:37.085388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:37.099585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339363f-af83-4f0f-bc90-eaa40addfa1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:37.100220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339363f-af83-4f0f-bc90-eaa40addfa1a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:37.100520 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339363f-af83-4f0f-bc90-eaa40addfa1a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:37.100772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339363f-af83-4f0f-bc90-eaa40addfa1a None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:37.101471 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4339363f-af83-4f0f-bc90-eaa40addfa1a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:37.101785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4339363f-af83-4f0f-bc90-eaa40addfa1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:37.103843 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4339363f-af83-4f0f-bc90-eaa40addfa1a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:37.113045 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 148/283] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Sep 1 16:17:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:37.113920 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339363f-af83-4f0f-bc90-eaa40addfa1a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:37.156416 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 136/284] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:37.157082 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc8a581-9928-499e-887b-4af4cfb119a6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:37.202871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:37.247022 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339363f-af83-4f0f-bc90-eaa40addfa1a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:37.310720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8891bb8-5fd7-46d5-bf1b-9a70135d1b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:37.311211 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc8a581-9928-499e-887b-4af4cfb119a6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:37.311591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8891bb8-5fd7-46d5-bf1b-9a70135d1b6e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:37.311975 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8891bb8-5fd7-46d5-bf1b-9a70135d1b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:37.312349 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8891bb8-5fd7-46d5-bf1b-9a70135d1b6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:37.313301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8891bb8-5fd7-46d5-bf1b-9a70135d1b6e None admin] RBAC: Adding query filter params (role.id=42db5f955f2644b1bb22b0a68f8cd829, user.id=e179ef144e1147faad8f3e61bc0a8e88, scope.system=all) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:37.314215 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8891bb8-5fd7-46d5-bf1b-9a70135d1b6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:37.315361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8891bb8-5fd7-46d5-bf1b-9a70135d1b6e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:37.320927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 149/285] 10.222.0.58 () {56 vars in 1308 bytes} [Thu Sep 1 16:17:37 2022] GET /identity/v3/role_assignments?user.id=e179ef144e1147faad8f3e61bc0a8e88&scope.system=all&role.id=42db5f955f2644b1bb22b0a68f8cd829 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:37.350777 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce7aff-a636-43d1-992d-fdbefed19772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:37.351455 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce7aff-a636-43d1-992d-fdbefed19772 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:37.351740 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce7aff-a636-43d1-992d-fdbefed19772 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:37.352021 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce7aff-a636-43d1-992d-fdbefed19772 None admin] PATH_INFO: `/v3/users/89bcb151185747288af5f6dcee50db27` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:37.353307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ce7aff-a636-43d1-992d-fdbefed19772 None admin] RBAC: Authorizing `identity:get_user(user_id=89bcb151185747288af5f6dcee50db27)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:37.354832 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ce7aff-a636-43d1-992d-fdbefed19772 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:37.356645 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 137/286] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:37 2022] GET /identity/v3/users/89bcb151185747288af5f6dcee50db27 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:37.362990 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8891bb8-5fd7-46d5-bf1b-9a70135d1b6e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:37.427707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:37.451991 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce7aff-a636-43d1-992d-fdbefed19772 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:37.491316 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8891bb8-5fd7-46d5-bf1b-9a70135d1b6e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:37.537503 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123d7d0b-d0d6-4d23-9a20-11739cdf1849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:37.538225 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d7d0b-d0d6-4d23-9a20-11739cdf1849 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:37.539786 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d7d0b-d0d6-4d23-9a20-11739cdf1849 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:37.540069 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d7d0b-d0d6-4d23-9a20-11739cdf1849 None admin] PATH_INFO: `/v3/projects/2375c71530404fc591cb968d121ae21e` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:37.542596 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123d7d0b-d0d6-4d23-9a20-11739cdf1849 None admin] RBAC: Authorizing `identity:get_project(project_id=2375c71530404fc591cb968d121ae21e)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:37.544898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123d7d0b-d0d6-4d23-9a20-11739cdf1849 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:37.546941 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 150/287] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:37 2022] GET /identity/v3/projects/2375c71530404fc591cb968d121ae21e => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:37.554666 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123d7d0b-d0d6-4d23-9a20-11739cdf1849 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:37.575931 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:37.585817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce7aff-a636-43d1-992d-fdbefed19772 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:37.641902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09757b30-4d9a-4fb6-aa5d-7cf49e9c5edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:37.642568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09757b30-4d9a-4fb6-aa5d-7cf49e9c5edf None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:37.642850 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09757b30-4d9a-4fb6-aa5d-7cf49e9c5edf None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:37.643123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09757b30-4d9a-4fb6-aa5d-7cf49e9c5edf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:37.666168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09757b30-4d9a-4fb6-aa5d-7cf49e9c5edf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:37.671361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-09757b30-4d9a-4fb6-aa5d-7cf49e9c5edf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:37.671361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:37.671361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:37.671361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:37.671361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:37.671361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:37.671361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:37.671361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:37.671361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:37.671361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:37.671361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:37.671361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:37.671361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:37.671361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:37.671361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:37.671361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:37.672449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:37.672449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:37.672449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:37.672449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:37.672449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:37.672449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:37.672449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:37.672449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:37.672449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:37.672449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:37.672449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:37.672449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:37.672449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:37.672449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:37.672449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:37.672449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:37.673738 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:37.673738 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:37.673738 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:37.673738 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:37.673738 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:37.673738 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:37.673738 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:37.673738 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:37.673738 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:37.673738 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:37.673738 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:37.673738 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:17:37.673738 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 138/288] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:37.679423 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09757b30-4d9a-4fb6-aa5d-7cf49e9c5edf None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:37.787910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123d7d0b-d0d6-4d23-9a20-11739cdf1849 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:37.817509 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:37.839778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775cd244-b381-4846-97a4-b61559318598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:37.840610 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775cd244-b381-4846-97a4-b61559318598 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:37.841425 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775cd244-b381-4846-97a4-b61559318598 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:37.841726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775cd244-b381-4846-97a4-b61559318598 None admin] PATH_INFO: `/v3/projects/2375c71530404fc591cb968d121ae21e/users/89bcb151185747288af5f6dcee50db27/roles/d657364ebb9c4110bc1acfe8a9f2f3fd` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:37.843752 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775cd244-b381-4846-97a4-b61559318598 None admin] RBAC: Authorizing `identity:create_grant(project_id=2375c71530404fc591cb968d121ae21e, user_id=89bcb151185747288af5f6dcee50db27, role_id=d657364ebb9c4110bc1acfe8a9f2f3fd)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:37.845282 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775cd244-b381-4846-97a4-b61559318598 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:37.875570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 151/289] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Sep 1 16:17:37 2022] PUT /identity/v3/projects/2375c71530404fc591cb968d121ae21e/users/89bcb151185747288af5f6dcee50db27/roles/d657364ebb9c4110bc1acfe8a9f2f3fd => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:17:37.878374 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09757b30-4d9a-4fb6-aa5d-7cf49e9c5edf None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:37.918863 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3429f08-1883-4e71-8a9c-5e46e16c1327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:37.919476 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3429f08-1883-4e71-8a9c-5e46e16c1327 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:37.919775 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3429f08-1883-4e71-8a9c-5e46e16c1327 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:37.920026 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3429f08-1883-4e71-8a9c-5e46e16c1327 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:37.920708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3429f08-1883-4e71-8a9c-5e46e16c1327 None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:37.921020 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3429f08-1883-4e71-8a9c-5e46e16c1327 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:37.922524 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3429f08-1883-4e71-8a9c-5e46e16c1327 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:37.947191 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 139/290] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Sep 1 16:17:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:37.952809 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775cd244-b381-4846-97a4-b61559318598 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:38.115028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775cd244-b381-4846-97a4-b61559318598 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:38.160848 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d866fb-b864-4450-be0e-4a5532ad8792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:38.161489 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d866fb-b864-4450-be0e-4a5532ad8792 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:38.161773 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d866fb-b864-4450-be0e-4a5532ad8792 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:38.162050 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d866fb-b864-4450-be0e-4a5532ad8792 None admin] PATH_INFO: `/v3/users/da4dd935f69048948da827011e300d74` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:38.163237 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d866fb-b864-4450-be0e-4a5532ad8792 None admin] RBAC: Authorizing `identity:get_user(user_id=da4dd935f69048948da827011e300d74)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:38.164682 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d866fb-b864-4450-be0e-4a5532ad8792 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:38.166897 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 152/291] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:37 2022] GET /identity/v3/users/da4dd935f69048948da827011e300d74 => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:38.175271 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3429f08-1883-4e71-8a9c-5e46e16c1327 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:38.210153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:38.351387 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3429f08-1883-4e71-8a9c-5e46e16c1327 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:38.400420 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f4ae2-d803-4bd7-96bc-f0a15cd20cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:38.401034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f4ae2-d803-4bd7-96bc-f0a15cd20cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:38.401320 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f4ae2-d803-4bd7-96bc-f0a15cd20cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:38.401599 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f4ae2-d803-4bd7-96bc-f0a15cd20cf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:38.518169 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3f4ae2-d803-4bd7-96bc-f0a15cd20cf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:38.519686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3f4ae2-d803-4bd7-96bc-f0a15cd20cf6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:38.544618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:38.642780 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 140/292] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:17:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:17:38.648394 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d866fb-b864-4450-be0e-4a5532ad8792 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:38.708805 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:38.782125 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d866fb-b864-4450-be0e-4a5532ad8792 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:38.828887 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a977d5f8-8f84-4667-9372-56cfb44508c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:38.829544 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a977d5f8-8f84-4667-9372-56cfb44508c9 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:38.829791 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a977d5f8-8f84-4667-9372-56cfb44508c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:38.830070 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a977d5f8-8f84-4667-9372-56cfb44508c9 None admin] PATH_INFO: `/v3/projects/2375c71530404fc591cb968d121ae21e` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:38.835059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a977d5f8-8f84-4667-9372-56cfb44508c9 None admin] RBAC: Authorizing `identity:get_project(project_id=2375c71530404fc591cb968d121ae21e)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:38.836537 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a977d5f8-8f84-4667-9372-56cfb44508c9 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:38.838346 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 153/293] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:38 2022] GET /identity/v3/projects/2375c71530404fc591cb968d121ae21e => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:38.844338 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f4ae2-d803-4bd7-96bc-f0a15cd20cf6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:38.966542 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f4ae2-d803-4bd7-96bc-f0a15cd20cf6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:38.985904 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:39.004051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145ae7a-f577-41f0-8f0b-6cc3a533a29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:39.004658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145ae7a-f577-41f0-8f0b-6cc3a533a29d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:39.004942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145ae7a-f577-41f0-8f0b-6cc3a533a29d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:39.005215 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145ae7a-f577-41f0-8f0b-6cc3a533a29d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:39.006003 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145ae7a-f577-41f0-8f0b-6cc3a533a29d None admin] RBAC: Adding query filter params (role.id=42db5f955f2644b1bb22b0a68f8cd829, scope.project.id=2375c71530404fc591cb968d121ae21e, user.id=da4dd935f69048948da827011e300d74) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:39.006351 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145ae7a-f577-41f0-8f0b-6cc3a533a29d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:39.007840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145ae7a-f577-41f0-8f0b-6cc3a533a29d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:39.015024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 141/294] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:17:38 2022] GET /identity/v3/role_assignments?user.id=da4dd935f69048948da827011e300d74&scope.project.id=2375c71530404fc591cb968d121ae21e&role.id=42db5f955f2644b1bb22b0a68f8cd829 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:39.542297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbab8a2-b382-4f8c-a445-d1c9f5aa02b6 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:39.542672 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbab8a2-b382-4f8c-a445-d1c9f5aa02b6 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:39.542856 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbab8a2-b382-4f8c-a445-d1c9f5aa02b6 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:39.544556 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 154/295] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:39.549822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78983e6b-a559-412a-9a6f-18058230fe25 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:39.550119 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78983e6b-a559-412a-9a6f-18058230fe25 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:39.550423 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78983e6b-a559-412a-9a6f-18058230fe25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:39.570311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-78983e6b-a559-412a-9a6f-18058230fe25 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:39.572024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-78983e6b-a559-412a-9a6f-18058230fe25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:39.654614 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 142/296] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:17:39.681449 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294c80d-8f18-42a2-afc4-ab0fda379703 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:39.681744 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294c80d-8f18-42a2-afc4-ab0fda379703 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:39.681997 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294c80d-8f18-42a2-afc4-ab0fda379703 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:39.683692 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 155/297] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:39.691472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82fc8ed-811e-4098-a161-51501fc00c94 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:39.691766 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82fc8ed-811e-4098-a161-51501fc00c94 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:39.692047 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82fc8ed-811e-4098-a161-51501fc00c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:39.710382 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a82fc8ed-811e-4098-a161-51501fc00c94 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:39.711687 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a82fc8ed-811e-4098-a161-51501fc00c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:39.780643 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 143/298] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:39.918369 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a1973-aaa8-45e9-a828-c3c506231b05 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:39.918998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a1973-aaa8-45e9-a828-c3c506231b05 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:39.918998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a1973-aaa8-45e9-a828-c3c506231b05 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:39.920530 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 156/299] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:39.925671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678160d-c7fd-4bd5-b50a-30ac75289f68 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:39.925964 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678160d-c7fd-4bd5-b50a-30ac75289f68 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:39.926270 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678160d-c7fd-4bd5-b50a-30ac75289f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:39.944832 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7678160d-c7fd-4bd5-b50a-30ac75289f68 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:39.945363 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a1973-aaa8-45e9-a828-c3c506231b05 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:39.946015 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7678160d-c7fd-4bd5-b50a-30ac75289f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:39.988016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:40.030291 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 144/300] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:40.057008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7678160d-c7fd-4bd5-b50a-30ac75289f68 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:40.073128 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a1973-aaa8-45e9-a828-c3c506231b05 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:40.090004 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:40.116137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7f65d-b54b-42ee-ba74-48708b322161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:40.116766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7f65d-b54b-42ee-ba74-48708b322161 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:40.117039 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7f65d-b54b-42ee-ba74-48708b322161 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:40.117302 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7f65d-b54b-42ee-ba74-48708b322161 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:40.132583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7f65d-b54b-42ee-ba74-48708b322161 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:40.137395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-5ee7f65d-b54b-42ee-ba74-48708b322161 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:17:40.137395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:40.137395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:40.137395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:40.137395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:40.137395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:40.137395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:40.137395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:40.137395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:40.137395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:40.137395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:40.137395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:40.137395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:40.137395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:40.137395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:40.137395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:40.138361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:40.138361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:40.138361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:40.138361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:40.138361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:40.138361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:40.138361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:40.138361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:40.138361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:40.138361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:40.138361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:40.138361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:40.138361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:40.138361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:40.138361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:40.138361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:40.139190 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:40.139190 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:40.139190 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:40.139190 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:40.139190 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:40.139190 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:40.139190 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:40.139190 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:40.139190 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:40.139190 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:40.139190 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:17:40.139190 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:17:40.142327 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 157/301] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:17:40.143878 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f1285-feee-49a8-84d7-7de2638ce9ee None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:40.144170 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f1285-feee-49a8-84d7-7de2638ce9ee None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:40.144446 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f1285-feee-49a8-84d7-7de2638ce9ee None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:40.146258 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 158/302] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:40.146892 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f1285-feee-49a8-84d7-7de2638ce9ee None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:40.175795 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7678160d-c7fd-4bd5-b50a-30ac75289f68 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:40.214025 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d091254-a0d7-4a91-aaf8-bb06bc4d8cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:40.214604 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d091254-a0d7-4a91-aaf8-bb06bc4d8cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:40.214886 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d091254-a0d7-4a91-aaf8-bb06bc4d8cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:40.215308 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d091254-a0d7-4a91-aaf8-bb06bc4d8cd2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:40.216244 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d091254-a0d7-4a91-aaf8-bb06bc4d8cd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:40.217688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d091254-a0d7-4a91-aaf8-bb06bc4d8cd2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:40.219702 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 145/303] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:17:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:40.220757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cde226-5282-437e-ad1e-e6ca97e62ee7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:40.221039 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cde226-5282-437e-ad1e-e6ca97e62ee7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:40.221395 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cde226-5282-437e-ad1e-e6ca97e62ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:40.241042 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-70cde226-5282-437e-ad1e-e6ca97e62ee7 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:40.242261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-70cde226-5282-437e-ad1e-e6ca97e62ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:40.260935 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f1285-feee-49a8-84d7-7de2638ce9ee None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:40.273854 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:40.298926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ec778-a81f-4226-8b9f-4b809964ec0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:40.299614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ec778-a81f-4226-8b9f-4b809964ec0e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:40.299897 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ec778-a81f-4226-8b9f-4b809964ec0e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:40.300166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ec778-a81f-4226-8b9f-4b809964ec0e None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:40.300834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6ec778-a81f-4226-8b9f-4b809964ec0e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:40.301141 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6ec778-a81f-4226-8b9f-4b809964ec0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:40.302537 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6ec778-a81f-4226-8b9f-4b809964ec0e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:40.311996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 159/304] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Sep 1 16:17:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:40.312783 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ec778-a81f-4226-8b9f-4b809964ec0e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:40.315045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 146/305] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:40.315724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cde226-5282-437e-ad1e-e6ca97e62ee7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:40.355377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:40.422892 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ec778-a81f-4226-8b9f-4b809964ec0e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:40.426741 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cde226-5282-437e-ad1e-e6ca97e62ee7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:40.457334 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f0bbb-5136-4611-a058-812b21e4d902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:40.457934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f0bbb-5136-4611-a058-812b21e4d902 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:40.458235 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f0bbb-5136-4611-a058-812b21e4d902 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:40.458516 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f0bbb-5136-4611-a058-812b21e4d902 None admin] PATH_INFO: `/v3/groups` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:40.459140 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427f0bbb-5136-4611-a058-812b21e4d902 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:40.460558 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427f0bbb-5136-4611-a058-812b21e4d902 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:40.463761 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13519d95-a52b-4176-a714-b3a20f10c920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:40.464733 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13519d95-a52b-4176-a714-b3a20f10c920 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:40.465321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13519d95-a52b-4176-a714-b3a20f10c920 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:40.465814 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13519d95-a52b-4176-a714-b3a20f10c920 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:40.480154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13519d95-a52b-4176-a714-b3a20f10c920 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:40.480951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 160/306] 10.222.0.58 () {60 vars in 1134 bytes} [Thu Sep 1 16:17:40 2022] POST /identity/v3/groups => generated 229 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:17:40.481946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f0bbb-5136-4611-a058-812b21e4d902 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:40.485847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-13519d95-a52b-4176-a714-b3a20f10c920 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:40.485847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:40.485847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:40.485847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:40.485847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:40.485847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:40.485847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:40.485847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:40.485847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:40.485847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:40.485847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:40.485847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:40.485847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:40.485847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:40.485847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:40.485847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:40.486894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:40.486894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:40.486894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:40.486894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:40.486894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:40.486894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:40.486894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:40.486894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:40.486894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:40.486894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:40.486894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:40.486894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:40.486894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:40.486894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:40.486894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:40.486894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:40.487990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:40.487990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:40.487990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:40.487990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:40.487990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:40.487990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:40.487990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:40.487990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:40.487990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:40.487990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:40.487990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:40.487990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:17:40.487990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 147/307] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:40.493148 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13519d95-a52b-4176-a714-b3a20f10c920 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:40.532234 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:40.618305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13519d95-a52b-4176-a714-b3a20f10c920 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:40.626501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:40.635380 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f0bbb-5136-4611-a058-812b21e4d902 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:40.654934 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db519679-a8e4-42a2-b480-a17bc5974088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:40.655572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db519679-a8e4-42a2-b480-a17bc5974088 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:40.655826 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db519679-a8e4-42a2-b480-a17bc5974088 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:40.656100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db519679-a8e4-42a2-b480-a17bc5974088 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:40.657157 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db519679-a8e4-42a2-b480-a17bc5974088 None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:40.657157 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db519679-a8e4-42a2-b480-a17bc5974088 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:40.658531 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db519679-a8e4-42a2-b480-a17bc5974088 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:40.669224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 148/308] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Sep 1 16:17:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:40.670051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db519679-a8e4-42a2-b480-a17bc5974088 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:40.679156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1608b7-3b11-4a52-8d7c-ef116540b35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:40.679156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1608b7-3b11-4a52-8d7c-ef116540b35c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:40.679369 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1608b7-3b11-4a52-8d7c-ef116540b35c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:40.679569 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1608b7-3b11-4a52-8d7c-ef116540b35c None admin] PATH_INFO: `/v3/users/d3f84838faef4437895581a402fa2e29` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:40.696143 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1608b7-3b11-4a52-8d7c-ef116540b35c None admin] RBAC: Authorizing `identity:get_user(user_id=d3f84838faef4437895581a402fa2e29)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:40.697657 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1608b7-3b11-4a52-8d7c-ef116540b35c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:40.699659 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 161/309] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:40 2022] GET /identity/v3/users/d3f84838faef4437895581a402fa2e29 => generated 293 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:40.700418 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1608b7-3b11-4a52-8d7c-ef116540b35c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:40.762031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:40.772463 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db519679-a8e4-42a2-b480-a17bc5974088 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:40.807153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1608b7-3b11-4a52-8d7c-ef116540b35c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:40.807362 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7948af9-df59-4fff-ab94-f6de9d27f98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:40.807861 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7948af9-df59-4fff-ab94-f6de9d27f98a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:40.808142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7948af9-df59-4fff-ab94-f6de9d27f98a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:40.808473 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7948af9-df59-4fff-ab94-f6de9d27f98a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:40.818967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7948af9-df59-4fff-ab94-f6de9d27f98a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:40.824913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-c7948af9-df59-4fff-ab94-f6de9d27f98a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:17:40.824913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:40.824913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:40.824913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:40.824913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:40.824913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:40.824913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:40.824913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:40.824913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:40.824913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:40.824913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:40.824913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:40.824913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:40.824913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:40.824913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:40.824913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:40.825824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:40.825824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:40.825824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:40.825824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:40.825824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:40.825824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:40.825824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:40.825824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:40.825824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:40.825824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:40.825824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:40.825824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:40.825824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:40.825824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:40.825824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:40.825824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:40.831224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:40.831224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:40.831224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:40.831224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:40.831224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:40.831224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:40.831224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:40.831224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:40.831224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:40.831224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:40.831224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:17:40.831224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:17:40.831224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 149/310] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:40.831224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7948af9-df59-4fff-ab94-f6de9d27f98a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:40.850424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c88f7f3-5ce6-4390-87eb-de68d96d9151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:40.851060 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88f7f3-5ce6-4390-87eb-de68d96d9151 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:40.851342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88f7f3-5ce6-4390-87eb-de68d96d9151 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:40.851615 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88f7f3-5ce6-4390-87eb-de68d96d9151 None admin] PATH_INFO: `/v3/users/e179ef144e1147faad8f3e61bc0a8e88` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:40.853742 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c88f7f3-5ce6-4390-87eb-de68d96d9151 None admin] RBAC: Authorizing `identity:get_user(user_id=e179ef144e1147faad8f3e61bc0a8e88)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:40.855318 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c88f7f3-5ce6-4390-87eb-de68d96d9151 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:40.857159 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 162/311] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:40 2022] GET /identity/v3/users/e179ef144e1147faad8f3e61bc0a8e88 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:40.857919 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c88f7f3-5ce6-4390-87eb-de68d96d9151 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:40.882055 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:40.940069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7948af9-df59-4fff-ab94-f6de9d27f98a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:40.970794 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c88f7f3-5ce6-4390-87eb-de68d96d9151 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:40.977674 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c656a0-1ebf-47f8-b907-07783afacf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:40.978401 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c656a0-1ebf-47f8-b907-07783afacf94 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:40.978794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c656a0-1ebf-47f8-b907-07783afacf94 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:40.979126 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c656a0-1ebf-47f8-b907-07783afacf94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:40.979966 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c656a0-1ebf-47f8-b907-07783afacf94 None admin] RBAC: Adding query filter params (role.id=d657364ebb9c4110bc1acfe8a9f2f3fd, user.id=d3f84838faef4437895581a402fa2e29, scope.system=all) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:40.980335 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c656a0-1ebf-47f8-b907-07783afacf94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:40.981852 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c656a0-1ebf-47f8-b907-07783afacf94 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:40.987550 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 150/312] 10.222.0.58 () {56 vars in 1308 bytes} [Thu Sep 1 16:17:40 2022] GET /identity/v3/role_assignments?user.id=d3f84838faef4437895581a402fa2e29&scope.system=all&role.id=d657364ebb9c4110bc1acfe8a9f2f3fd => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:40.989157 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c656a0-1ebf-47f8-b907-07783afacf94 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:41.010735 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d847f48-3bc8-45f8-87f8-16d6be5a4a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:41.011364 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d847f48-3bc8-45f8-87f8-16d6be5a4a32 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:41.011646 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d847f48-3bc8-45f8-87f8-16d6be5a4a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:41.011920 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d847f48-3bc8-45f8-87f8-16d6be5a4a32 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:41.012622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d847f48-3bc8-45f8-87f8-16d6be5a4a32 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:41.012927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d847f48-3bc8-45f8-87f8-16d6be5a4a32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:41.014411 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d847f48-3bc8-45f8-87f8-16d6be5a4a32 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:41.020884 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:41.030317 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 163/313] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Sep 1 16:17:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:41.031341 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d847f48-3bc8-45f8-87f8-16d6be5a4a32 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:41.129342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:41.152852 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c656a0-1ebf-47f8-b907-07783afacf94 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:41.159593 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d847f48-3bc8-45f8-87f8-16d6be5a4a32 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:41.200197 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe983f-ee29-494f-a25d-fdd28e32d38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:41.200910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe983f-ee29-494f-a25d-fdd28e32d38a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:41.201283 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe983f-ee29-494f-a25d-fdd28e32d38a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:41.201649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe983f-ee29-494f-a25d-fdd28e32d38a None admin] PATH_INFO: `/v3/users/89bcb151185747288af5f6dcee50db27` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:41.202279 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb3abd-77d9-422b-9a9d-6b041432c0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:41.202941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb3abd-77d9-422b-9a9d-6b041432c0b3 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:41.203226 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb3abd-77d9-422b-9a9d-6b041432c0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:41.203509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb3abd-77d9-422b-9a9d-6b041432c0b3 None admin] PATH_INFO: `/v3/system/users/e179ef144e1147faad8f3e61bc0a8e88/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:41.203741 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe983f-ee29-494f-a25d-fdd28e32d38a None admin] RBAC: Authorizing `identity:get_user(user_id=89bcb151185747288af5f6dcee50db27)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:41.204928 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fb3abd-77d9-422b-9a9d-6b041432c0b3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e179ef144e1147faad8f3e61bc0a8e88, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:41.205615 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe983f-ee29-494f-a25d-fdd28e32d38a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:41.206420 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fb3abd-77d9-422b-9a9d-6b041432c0b3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:41.208236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 164/314] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:41 2022] GET /identity/v3/users/89bcb151185747288af5f6dcee50db27 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:41.213714 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe983f-ee29-494f-a25d-fdd28e32d38a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:41.221804 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 151/315] 10.222.0.58 () {58 vars in 1334 bytes} [Thu Sep 1 16:17:40 2022] PUT /identity/v3/system/users/e179ef144e1147faad8f3e61bc0a8e88/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:17:41.331760 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe983f-ee29-494f-a25d-fdd28e32d38a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:41.369965 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4bd8a8-8529-4f9f-9774-73f770791e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:41.371013 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4bd8a8-8529-4f9f-9774-73f770791e5c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:41.371488 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4bd8a8-8529-4f9f-9774-73f770791e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:41.371951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4bd8a8-8529-4f9f-9774-73f770791e5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:41.392315 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:41.450015 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4bd8a8-8529-4f9f-9774-73f770791e5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:41.451562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4bd8a8-8529-4f9f-9774-73f770791e5c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:41.557192 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 165/316] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:17:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 16:17:41.563541 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb3abd-77d9-422b-9a9d-6b041432c0b3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:41.648705 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:41.673420 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb3abd-77d9-422b-9a9d-6b041432c0b3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:41.709330 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d70b9-ace2-4a74-8057-82e17ee5c798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:41.709896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d70b9-ace2-4a74-8057-82e17ee5c798 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:41.710185 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d70b9-ace2-4a74-8057-82e17ee5c798 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:41.710494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d70b9-ace2-4a74-8057-82e17ee5c798 None admin] PATH_INFO: `/v3/projects/2375c71530404fc591cb968d121ae21e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:41.711618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d70b9-ace2-4a74-8057-82e17ee5c798 None admin] RBAC: Authorizing `identity:get_project(project_id=2375c71530404fc591cb968d121ae21e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:41.713641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d70b9-ace2-4a74-8057-82e17ee5c798 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:41.715590 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 152/317] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:41 2022] GET /identity/v3/projects/2375c71530404fc591cb968d121ae21e => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:41.723298 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4bd8a8-8529-4f9f-9774-73f770791e5c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:41.774253 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:41.826230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4bd8a8-8529-4f9f-9774-73f770791e5c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:41.862054 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff29d9-2fb9-473f-981d-a4e811e75563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:41.863063 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff29d9-2fb9-473f-981d-a4e811e75563 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:41.863568 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff29d9-2fb9-473f-981d-a4e811e75563 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:41.864029 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff29d9-2fb9-473f-981d-a4e811e75563 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:41.864945 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ff29d9-2fb9-473f-981d-a4e811e75563 None admin] RBAC: Adding query filter params (role.id=d657364ebb9c4110bc1acfe8a9f2f3fd, scope.project.id=2375c71530404fc591cb968d121ae21e, user.id=89bcb151185747288af5f6dcee50db27) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:41.865472 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ff29d9-2fb9-473f-981d-a4e811e75563 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:41.867123 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ff29d9-2fb9-473f-981d-a4e811e75563 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:41.874305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 166/318] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:17:41 2022] GET /identity/v3/role_assignments?user.id=89bcb151185747288af5f6dcee50db27&scope.project.id=2375c71530404fc591cb968d121ae21e&role.id=d657364ebb9c4110bc1acfe8a9f2f3fd => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:42.466596 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f976a-c5ae-4a77-af19-a41015870f84 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:42.466898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f976a-c5ae-4a77-af19-a41015870f84 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:42.467158 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f976a-c5ae-4a77-af19-a41015870f84 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:42.468691 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 153/319] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:42.473870 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d703e4-3349-48cc-9d82-c689e63f3d65 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:42.474329 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d703e4-3349-48cc-9d82-c689e63f3d65 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:42.474529 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d703e4-3349-48cc-9d82-c689e63f3d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:42.492478 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-98d703e4-3349-48cc-9d82-c689e63f3d65 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:42.493657 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-98d703e4-3349-48cc-9d82-c689e63f3d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:42.560462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 167/320] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:42.822039 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f976a-c5ae-4a77-af19-a41015870f84 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:42.926691 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f976a-c5ae-4a77-af19-a41015870f84 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:42.959511 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08034234-f82a-431d-8338-516c13d84a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:42.960134 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08034234-f82a-431d-8338-516c13d84a17 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:42.960410 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08034234-f82a-431d-8338-516c13d84a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:42.960679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08034234-f82a-431d-8338-516c13d84a17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:42.961755 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08034234-f82a-431d-8338-516c13d84a17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:42.963782 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08034234-f82a-431d-8338-516c13d84a17 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:42.965958 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 154/321] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:17:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:42.970988 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d703e4-3349-48cc-9d82-c689e63f3d65 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:42.981818 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffa8f1-923c-4564-8c94-7430e6faf803 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:42.982109 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffa8f1-923c-4564-8c94-7430e6faf803 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:42.982430 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffa8f1-923c-4564-8c94-7430e6faf803 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:42.983909 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 155/322] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:17:42.988763 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51036230-3ee3-49c3-845b-db4078d7abbf None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:42.989050 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51036230-3ee3-49c3-845b-db4078d7abbf None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:42.989328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51036230-3ee3-49c3-845b-db4078d7abbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:43.008091 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-51036230-3ee3-49c3-845b-db4078d7abbf None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:43.009887 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-51036230-3ee3-49c3-845b-db4078d7abbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:43.019855 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:43.024166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:43.087974 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d703e4-3349-48cc-9d82-c689e63f3d65 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:43.090228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 156/323] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:17:43.123871 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290fb87-abdd-4ce6-872b-ab9fb3e49103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:43.124578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290fb87-abdd-4ce6-872b-ab9fb3e49103 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:43.124860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290fb87-abdd-4ce6-872b-ab9fb3e49103 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:43.125128 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290fb87-abdd-4ce6-872b-ab9fb3e49103 None admin] PATH_INFO: `/v3/groups` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:43.125757 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4290fb87-abdd-4ce6-872b-ab9fb3e49103 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:43.127191 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4290fb87-abdd-4ce6-872b-ab9fb3e49103 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:43.143099 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 168/324] 10.222.0.58 () {60 vars in 1134 bytes} [Thu Sep 1 16:17:42 2022] POST /identity/v3/groups => generated 226 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:17:43.241656 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b47933-6368-495c-bde4-9e617f278068 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:43.241985 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b47933-6368-495c-bde4-9e617f278068 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:43.242219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b47933-6368-495c-bde4-9e617f278068 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:43.244350 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 157/325] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:43.249680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a3ee2-8c0b-49af-8ad9-2fdb6f4880fd None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:43.249965 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a3ee2-8c0b-49af-8ad9-2fdb6f4880fd None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:43.250268 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a3ee2-8c0b-49af-8ad9-2fdb6f4880fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:43.268861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-220a3ee2-8c0b-49af-8ad9-2fdb6f4880fd None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:43.270069 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-220a3ee2-8c0b-49af-8ad9-2fdb6f4880fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:43.347834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 169/326] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:43.359106 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b47933-6368-495c-bde4-9e617f278068 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:43.479404 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b47933-6368-495c-bde4-9e617f278068 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:43.517237 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed62375-8897-46a3-8586-fe0d32cf8485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:43.518238 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed62375-8897-46a3-8586-fe0d32cf8485 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:43.518701 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed62375-8897-46a3-8586-fe0d32cf8485 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:43.519149 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed62375-8897-46a3-8586-fe0d32cf8485 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:43.530131 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed62375-8897-46a3-8586-fe0d32cf8485 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:43.535411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-bed62375-8897-46a3-8586-fe0d32cf8485 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:17:43.535411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:43.535411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:43.535411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:43.535411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:43.535411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:43.535411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:43.535411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:43.535411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:43.535411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:43.535411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:43.535411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:43.535411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:43.535411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:43.535411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:43.535411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:43.536342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:43.536342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:43.536342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:43.536342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:43.536342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:43.536342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:43.536342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:43.536342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:43.536342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:43.536342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:43.536342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:43.536342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:43.536342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:43.536342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:43.536342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:43.536342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:43.537172 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:43.537172 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:43.537172 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:43.537172 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:43.537172 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:43.537172 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:43.537172 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:43.537172 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:43.537172 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:43.537172 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:43.537172 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:17:43.537172 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:17:43.539492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 158/327] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:43.546148 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220a3ee2-8c0b-49af-8ad9-2fdb6f4880fd None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:43.591032 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:43.619758 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed62375-8897-46a3-8586-fe0d32cf8485 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:43.626123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:43.674929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220a3ee2-8c0b-49af-8ad9-2fdb6f4880fd None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:43.713976 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb0162-95c5-4328-b432-08d425126192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:43.714623 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb0162-95c5-4328-b432-08d425126192 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:43.714918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb0162-95c5-4328-b432-08d425126192 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:43.715182 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb0162-95c5-4328-b432-08d425126192 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:43.715861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb0162-95c5-4328-b432-08d425126192 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:43.716155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb0162-95c5-4328-b432-08d425126192 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:43.717652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb0162-95c5-4328-b432-08d425126192 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:43.728890 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 170/328] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Sep 1 16:17:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:43.734534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb0162-95c5-4328-b432-08d425126192 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:43.765109 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed62375-8897-46a3-8586-fe0d32cf8485 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:43.809634 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048ff0e-5728-438d-a5a3-9c09ce35996d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:43.810593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048ff0e-5728-438d-a5a3-9c09ce35996d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:43.811052 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048ff0e-5728-438d-a5a3-9c09ce35996d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:43.811525 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048ff0e-5728-438d-a5a3-9c09ce35996d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:43.822908 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8048ff0e-5728-438d-a5a3-9c09ce35996d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:43.830822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-8048ff0e-5728-438d-a5a3-9c09ce35996d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:43.830822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:43.830822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:43.830822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:43.830822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:43.830822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:43.830822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:43.830822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:43.830822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:43.830822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:43.830822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:43.830822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:43.830822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:43.830822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:43.830822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:43.830822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:43.831742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:43.831742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:43.831742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:43.831742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:43.831742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:43.831742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:43.831742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:43.831742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:43.831742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:43.831742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:43.831742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:43.831742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:43.831742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:43.831742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:43.831742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:43.831742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:43.832546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:43.832546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:43.832546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:43.832546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:43.832546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:43.832546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:43.832546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:43.832546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:43.832546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:43.832546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:43.832546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:43.832546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:17:43.835149 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 159/329] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:43.840522 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048ff0e-5728-438d-a5a3-9c09ce35996d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:43.846856 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:43.855907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb0162-95c5-4328-b432-08d425126192 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:43.896298 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4f0e2-54e4-4cac-b53f-b3016c8e4da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:43.897315 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4f0e2-54e4-4cac-b53f-b3016c8e4da3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:43.897791 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4f0e2-54e4-4cac-b53f-b3016c8e4da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:43.898314 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4f0e2-54e4-4cac-b53f-b3016c8e4da3 None admin] PATH_INFO: `/v3/users/d3f84838faef4437895581a402fa2e29` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:43.899683 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d4f0e2-54e4-4cac-b53f-b3016c8e4da3 None admin] RBAC: Authorizing `identity:get_user(user_id=d3f84838faef4437895581a402fa2e29)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:43.901364 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d4f0e2-54e4-4cac-b53f-b3016c8e4da3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:43.903461 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 171/330] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:43 2022] GET /identity/v3/users/d3f84838faef4437895581a402fa2e29 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:43.909092 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4f0e2-54e4-4cac-b53f-b3016c8e4da3 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:43.926587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:43.963910 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048ff0e-5728-438d-a5a3-9c09ce35996d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:44.006074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1527e-53e6-48a3-887d-556a3c692bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:44.006715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1527e-53e6-48a3-887d-556a3c692bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:44.006997 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1527e-53e6-48a3-887d-556a3c692bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:44.007274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1527e-53e6-48a3-887d-556a3c692bb7 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:44.007942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e1527e-53e6-48a3-887d-556a3c692bb7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:44.008247 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e1527e-53e6-48a3-887d-556a3c692bb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:44.009671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e1527e-53e6-48a3-887d-556a3c692bb7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:44.019828 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 160/331] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Sep 1 16:17:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:44.025408 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1527e-53e6-48a3-887d-556a3c692bb7 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:44.067127 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4f0e2-54e4-4cac-b53f-b3016c8e4da3 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:44.134522 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878f25d-16e8-4573-ab2a-dcf3dd051368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:44.135175 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878f25d-16e8-4573-ab2a-dcf3dd051368 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:44.135453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878f25d-16e8-4573-ab2a-dcf3dd051368 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:44.135725 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878f25d-16e8-4573-ab2a-dcf3dd051368 None admin] PATH_INFO: `/v3/system/users/d3f84838faef4437895581a402fa2e29/roles/d657364ebb9c4110bc1acfe8a9f2f3fd` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:44.138164 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878f25d-16e8-4573-ab2a-dcf3dd051368 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d3f84838faef4437895581a402fa2e29, role_id=d657364ebb9c4110bc1acfe8a9f2f3fd)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:44.139548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878f25d-16e8-4573-ab2a-dcf3dd051368 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:44.160867 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 172/332] 10.222.0.58 () {58 vars in 1334 bytes} [Thu Sep 1 16:17:43 2022] PUT /identity/v3/system/users/d3f84838faef4437895581a402fa2e29/roles/d657364ebb9c4110bc1acfe8a9f2f3fd => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:17:44.182848 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1527e-53e6-48a3-887d-556a3c692bb7 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:44.211671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:44.223018 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f909b-8a25-4b13-b2a0-9d71e064f00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:44.223942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f909b-8a25-4b13-b2a0-9d71e064f00e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:44.224362 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f909b-8a25-4b13-b2a0-9d71e064f00e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:44.224805 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f909b-8a25-4b13-b2a0-9d71e064f00e None admin] PATH_INFO: `/v3/users/e179ef144e1147faad8f3e61bc0a8e88` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:44.226087 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f909b-8a25-4b13-b2a0-9d71e064f00e None admin] RBAC: Authorizing `identity:get_user(user_id=e179ef144e1147faad8f3e61bc0a8e88)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:44.228379 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f909b-8a25-4b13-b2a0-9d71e064f00e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:44.230501 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 161/333] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:44 2022] GET /identity/v3/users/e179ef144e1147faad8f3e61bc0a8e88 => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:44.236218 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878f25d-16e8-4573-ab2a-dcf3dd051368 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:44.274380 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:44.365102 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878f25d-16e8-4573-ab2a-dcf3dd051368 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:44.405398 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b59c3b-2a83-4173-ba75-0be9fcef3adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:44.406284 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b59c3b-2a83-4173-ba75-0be9fcef3adb None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:44.406742 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b59c3b-2a83-4173-ba75-0be9fcef3adb None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:44.407178 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b59c3b-2a83-4173-ba75-0be9fcef3adb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:44.408058 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b59c3b-2a83-4173-ba75-0be9fcef3adb None admin] RBAC: Adding query filter params (role.id=42db5f955f2644b1bb22b0a68f8cd829, user.id=e179ef144e1147faad8f3e61bc0a8e88, scope.system=all) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:44.408507 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b59c3b-2a83-4173-ba75-0be9fcef3adb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:44.410101 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b59c3b-2a83-4173-ba75-0be9fcef3adb None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:44.424746 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 173/334] 10.222.0.58 () {56 vars in 1308 bytes} [Thu Sep 1 16:17:44 2022] GET /identity/v3/role_assignments?user.id=e179ef144e1147faad8f3e61bc0a8e88&scope.system=all&role.id=42db5f955f2644b1bb22b0a68f8cd829 => generated 510 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:45.378137 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b85f55-dbc9-4d6e-be36-462eb61db60a None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:45.378636 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b85f55-dbc9-4d6e-be36-462eb61db60a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:45.378711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b85f55-dbc9-4d6e-be36-462eb61db60a None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:45.380325 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 162/335] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:45.386088 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2321f0-0ffd-41af-9ce1-eb1c28014d57 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:45.386088 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2321f0-0ffd-41af-9ce1-eb1c28014d57 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:45.386088 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2321f0-0ffd-41af-9ce1-eb1c28014d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:45.412270 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-9b2321f0-0ffd-41af-9ce1-eb1c28014d57 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:45.413811 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9b2321f0-0ffd-41af-9ce1-eb1c28014d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:45.508791 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 174/336] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:17:46.058142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b85f55-dbc9-4d6e-be36-462eb61db60a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:46.195915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b85f55-dbc9-4d6e-be36-462eb61db60a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:46.208995 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354635c-7a1b-4c29-816f-5d77760a88d9 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:46.209282 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354635c-7a1b-4c29-816f-5d77760a88d9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:46.209553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354635c-7a1b-4c29-816f-5d77760a88d9 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:46.211084 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 175/337] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:46.220796 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e6652-9a60-48eb-a869-74906908ea90 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:46.221080 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e6652-9a60-48eb-a869-74906908ea90 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:46.221355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e6652-9a60-48eb-a869-74906908ea90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:46.238074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55897af6-57e8-4c1e-b61f-e9b0f39725af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:46.238733 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55897af6-57e8-4c1e-b61f-e9b0f39725af None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:46.239014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55897af6-57e8-4c1e-b61f-e9b0f39725af None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:46.239287 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55897af6-57e8-4c1e-b61f-e9b0f39725af None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:46.256377 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55897af6-57e8-4c1e-b61f-e9b0f39725af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:46.261715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-55897af6-57e8-4c1e-b61f-e9b0f39725af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:46.261715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:46.261715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:46.261715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:46.261715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:46.261715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:46.261715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:46.261715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:46.261715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:46.261715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:46.261715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:46.261715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:46.261715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:46.261715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:46.261715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:46.261715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:46.262607 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:46.262607 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:46.262607 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:46.262607 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:46.262607 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:46.262607 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:46.262607 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:46.262607 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:46.262607 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:46.262607 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:46.262607 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:46.262607 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:46.262607 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:46.262607 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:46.262607 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:46.262607 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:46.263439 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:46.263439 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:46.263439 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:46.263439 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:46.263439 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:46.263439 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:46.263439 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:46.263439 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:46.263439 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:46.263439 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:46.263439 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:46.263439 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:17:46.263439 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 163/338] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:46.267219 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a01e6652-9a60-48eb-a869-74906908ea90 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:46.268520 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a01e6652-9a60-48eb-a869-74906908ea90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:46.268819 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55897af6-57e8-4c1e-b61f-e9b0f39725af None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:46.287731 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:46.371658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:46.387021 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 176/339] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:46.388280 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520f7b1-c5dd-42bc-8bd9-a28e1e66eef4 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:46.388563 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520f7b1-c5dd-42bc-8bd9-a28e1e66eef4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:46.388834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520f7b1-c5dd-42bc-8bd9-a28e1e66eef4 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:46.390075 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 177/340] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:46.401208 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba04da-7e79-4d84-8c29-e39b41d02ac0 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:46.401486 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba04da-7e79-4d84-8c29-e39b41d02ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:46.401755 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba04da-7e79-4d84-8c29-e39b41d02ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:46.441270 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-caba04da-7e79-4d84-8c29-e39b41d02ac0 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:46.442484 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-caba04da-7e79-4d84-8c29-e39b41d02ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:46.473342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55897af6-57e8-4c1e-b61f-e9b0f39725af None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:46.539640 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba6255-0ad2-440f-9332-2911bc686e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:46.540338 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba6255-0ad2-440f-9332-2911bc686e3d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:46.540620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba6255-0ad2-440f-9332-2911bc686e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:46.540892 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba6255-0ad2-440f-9332-2911bc686e3d None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:46.541563 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba6255-0ad2-440f-9332-2911bc686e3d None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:46.542120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba6255-0ad2-440f-9332-2911bc686e3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:46.543537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba6255-0ad2-440f-9332-2911bc686e3d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:46.559552 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 164/341] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Sep 1 16:17:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:46.564698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba6255-0ad2-440f-9332-2911bc686e3d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:46.566285 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 178/342] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:46.702299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba6255-0ad2-440f-9332-2911bc686e3d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:46.710746 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba04da-7e79-4d84-8c29-e39b41d02ac0 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:46.726340 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:46.756232 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0bcf5-068a-4c42-ada7-e3021aa05bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:46.756848 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0bcf5-068a-4c42-ada7-e3021aa05bdc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:46.757132 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0bcf5-068a-4c42-ada7-e3021aa05bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:46.757409 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0bcf5-068a-4c42-ada7-e3021aa05bdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:46.876279 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:46.907244 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a0bcf5-068a-4c42-ada7-e3021aa05bdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:46.909068 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a0bcf5-068a-4c42-ada7-e3021aa05bdc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:46.934917 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba04da-7e79-4d84-8c29-e39b41d02ac0 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:47.023013 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9984dc-1b26-4de2-a3ad-93a246db7138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:47.023585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9984dc-1b26-4de2-a3ad-93a246db7138 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:47.023869 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9984dc-1b26-4de2-a3ad-93a246db7138 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:47.024142 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9984dc-1b26-4de2-a3ad-93a246db7138 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:47.051540 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9984dc-1b26-4de2-a3ad-93a246db7138 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:47.056138 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-dc9984dc-1b26-4de2-a3ad-93a246db7138 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:47.056138 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:47.056138 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:47.056138 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:47.056138 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:47.056138 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:47.056138 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:47.056138 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:47.056138 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:47.056138 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:47.056138 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:47.056138 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:47.056138 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:47.056138 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:47.056138 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:47.056138 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:47.057056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:47.057056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:47.057056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:47.057056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:47.057056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:47.057056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:47.057056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:47.057056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:47.057056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:47.057056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:47.057056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:47.057056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:47.057056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:47.057056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:47.057056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:47.057056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:47.057891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:47.057891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:47.057891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:47.057891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:47.057891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:47.057891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:47.057891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:47.057891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:47.057891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:47.057891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:47.057891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:47.057891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:17:47.057891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 179/343] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:17:47.062469 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9984dc-1b26-4de2-a3ad-93a246db7138 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:47.072007 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 165/344] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:17:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:17:47.073423 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0bcf5-068a-4c42-ada7-e3021aa05bdc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:47.094507 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:47.208606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0bcf5-068a-4c42-ada7-e3021aa05bdc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:47.224846 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9984dc-1b26-4de2-a3ad-93a246db7138 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:47.255934 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148212f-3c1d-476b-860f-a0ffc57b8be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:47.256816 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148212f-3c1d-476b-860f-a0ffc57b8be0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:47.257476 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148212f-3c1d-476b-860f-a0ffc57b8be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:47.258356 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148212f-3c1d-476b-860f-a0ffc57b8be0 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:47.259152 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148212f-3c1d-476b-860f-a0ffc57b8be0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:47.259466 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148212f-3c1d-476b-860f-a0ffc57b8be0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:47.260893 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148212f-3c1d-476b-860f-a0ffc57b8be0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:47.270019 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 166/345] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Sep 1 16:17:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:47.271062 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148212f-3c1d-476b-860f-a0ffc57b8be0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:47.287480 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43931d8-73d7-4598-9426-abacf81bb2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:47.288476 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43931d8-73d7-4598-9426-abacf81bb2ee None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:47.289006 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43931d8-73d7-4598-9426-abacf81bb2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:47.289511 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43931d8-73d7-4598-9426-abacf81bb2ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:47.293487 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:47.302833 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43931d8-73d7-4598-9426-abacf81bb2ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:47.308804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-c43931d8-73d7-4598-9426-abacf81bb2ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:17:47.308804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:47.308804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:47.308804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:47.308804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:47.308804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:47.308804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:47.308804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:47.308804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:47.308804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:47.308804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:47.308804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:47.308804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:47.308804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:47.308804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:47.308804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:47.309759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:47.309759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:47.309759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:47.309759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:47.309759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:47.309759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:47.309759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:47.309759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:47.309759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:47.309759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:47.309759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:47.309759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:47.309759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:47.309759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:47.309759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:47.309759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:47.310722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:47.310722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:47.310722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:47.310722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:47.310722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:47.310722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:47.310722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:47.310722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:47.310722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:47.310722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:47.310722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:17:47.310722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:17:47.319559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 180/346] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Sep 1 16:17:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:47.322960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236eda4-c7b6-4299-a8b3-e8f4f3d362d8 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:47.323463 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236eda4-c7b6-4299-a8b3-e8f4f3d362d8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:47.323944 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236eda4-c7b6-4299-a8b3-e8f4f3d362d8 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:47.325775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 181/347] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:47.326715 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236eda4-c7b6-4299-a8b3-e8f4f3d362d8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:47.385874 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:47.414046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148212f-3c1d-476b-860f-a0ffc57b8be0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:47.460132 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c83ccd-abc0-4afe-875d-483f1d1a26fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:47.460771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c83ccd-abc0-4afe-875d-483f1d1a26fb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:47.461050 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c83ccd-abc0-4afe-875d-483f1d1a26fb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:47.461323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c83ccd-abc0-4afe-875d-483f1d1a26fb None admin] PATH_INFO: `/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:47.461616 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236eda4-c7b6-4299-a8b3-e8f4f3d362d8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:47.464503 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c83ccd-abc0-4afe-875d-483f1d1a26fb None admin] RBAC: Authorizing `identity:get_project(project_id=9ea4cc94eaf84d0c8fec375d056242eb)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:47.467176 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c83ccd-abc0-4afe-875d-483f1d1a26fb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:47.469038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 167/348] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:47 2022] GET /identity/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:47.469833 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c83ccd-abc0-4afe-875d-483f1d1a26fb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:47.519475 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a98d69-a1d1-4f68-bdf4-0fa50e4878f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:47.520219 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a98d69-a1d1-4f68-bdf4-0fa50e4878f6 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:47.520505 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a98d69-a1d1-4f68-bdf4-0fa50e4878f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:47.521021 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a98d69-a1d1-4f68-bdf4-0fa50e4878f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:47.607126 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:47.615876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a98d69-a1d1-4f68-bdf4-0fa50e4878f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:47.617470 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a98d69-a1d1-4f68-bdf4-0fa50e4878f6 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:47.621306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c83ccd-abc0-4afe-875d-483f1d1a26fb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:47.661311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44457722-9217-442b-9031-d292e929af51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:47.662224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44457722-9217-442b-9031-d292e929af51 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:47.663441 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44457722-9217-442b-9031-d292e929af51 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:47.663669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44457722-9217-442b-9031-d292e929af51 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:47.664336 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44457722-9217-442b-9031-d292e929af51 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:47.664637 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44457722-9217-442b-9031-d292e929af51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:47.666021 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44457722-9217-442b-9031-d292e929af51 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:47.674754 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 168/349] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:17:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:47.680277 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd367c9-9262-46a7-b10f-a1b1dfa729ff None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:47.680564 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd367c9-9262-46a7-b10f-a1b1dfa729ff None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:47.680839 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd367c9-9262-46a7-b10f-a1b1dfa729ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:47.700001 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7cd367c9-9262-46a7-b10f-a1b1dfa729ff None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:47.701204 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7cd367c9-9262-46a7-b10f-a1b1dfa729ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:47.703691 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:47.738301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 182/350] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:17:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:17:47.739405 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a98d69-a1d1-4f68-bdf4-0fa50e4878f6 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:47.793655 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 169/351] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:47.795460 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d4f9e-55f7-4f06-b705-2c9e63fd323a None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:47.795749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d4f9e-55f7-4f06-b705-2c9e63fd323a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:47.796024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d4f9e-55f7-4f06-b705-2c9e63fd323a None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:47.798321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 170/352] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:47.799287 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d4f9e-55f7-4f06-b705-2c9e63fd323a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:47.868667 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:47.875469 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a98d69-a1d1-4f68-bdf4-0fa50e4878f6 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:47.920827 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbdb526-b5a7-4441-8f5d-dca64147464e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:47.921908 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdb526-b5a7-4441-8f5d-dca64147464e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:47.922387 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdb526-b5a7-4441-8f5d-dca64147464e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:47.922688 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdb526-b5a7-4441-8f5d-dca64147464e None admin] PATH_INFO: `/v3/groups/109db92706954923b3c4932e0a7d9e5e` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:47.930102 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d4f9e-55f7-4f06-b705-2c9e63fd323a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:47.932421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbdb526-b5a7-4441-8f5d-dca64147464e None admin] RBAC: Authorizing `identity:get_group(group_id=109db92706954923b3c4932e0a7d9e5e)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:47.935094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbdb526-b5a7-4441-8f5d-dca64147464e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:47.937018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 183/353] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Sep 1 16:17:47 2022] GET /identity/v3/groups/109db92706954923b3c4932e0a7d9e5e => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:47.938064 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbdb526-b5a7-4441-8f5d-dca64147464e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:47.980846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39cd51-8838-484f-bb5d-eb9fa84a9a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:47.981836 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39cd51-8838-484f-bb5d-eb9fa84a9a23 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:47.982342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39cd51-8838-484f-bb5d-eb9fa84a9a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:47.982865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39cd51-8838-484f-bb5d-eb9fa84a9a23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:48.004693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:48.096531 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39cd51-8838-484f-bb5d-eb9fa84a9a23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:48.098022 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39cd51-8838-484f-bb5d-eb9fa84a9a23 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:48.154948 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbdb526-b5a7-4441-8f5d-dca64147464e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:48.200137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797cfd38-0505-4a2d-9aea-47fbac37aeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:48.200137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797cfd38-0505-4a2d-9aea-47fbac37aeb1 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:48.200421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797cfd38-0505-4a2d-9aea-47fbac37aeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:48.200763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797cfd38-0505-4a2d-9aea-47fbac37aeb1 None admin] PATH_INFO: `/v3/projects/2375c71530404fc591cb968d121ae21e` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:48.201798 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797cfd38-0505-4a2d-9aea-47fbac37aeb1 None admin] RBAC: Authorizing `identity:get_project(project_id=2375c71530404fc591cb968d121ae21e)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:48.203298 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797cfd38-0505-4a2d-9aea-47fbac37aeb1 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:48.206075 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 184/354] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:47 2022] GET /identity/v3/projects/2375c71530404fc591cb968d121ae21e => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:48.207384 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31318461-3321-45a6-9226-74cd594a11a3 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:48.207667 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31318461-3321-45a6-9226-74cd594a11a3 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:48.207938 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31318461-3321-45a6-9226-74cd594a11a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:48.231112 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-31318461-3321-45a6-9226-74cd594a11a3 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:48.231112 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-31318461-3321-45a6-9226-74cd594a11a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:48.251068 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 171/355] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:17:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:17:48.251938 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39cd51-8838-484f-bb5d-eb9fa84a9a23 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:48.260371 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:48.297555 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:48.315654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 185/356] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:48.318997 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31318461-3321-45a6-9226-74cd594a11a3 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:48.379198 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39cd51-8838-484f-bb5d-eb9fa84a9a23 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:48.431262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dfd030-dece-4fd7-884d-6e96dac47b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:48.431262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfd030-dece-4fd7-884d-6e96dac47b65 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:48.431262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfd030-dece-4fd7-884d-6e96dac47b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:48.431262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfd030-dece-4fd7-884d-6e96dac47b65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:48.432233 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dfd030-dece-4fd7-884d-6e96dac47b65 None admin] RBAC: Adding query filter params (group.id=109db92706954923b3c4932e0a7d9e5e, role.id=42db5f955f2644b1bb22b0a68f8cd829, scope.project.id=9ea4cc94eaf84d0c8fec375d056242eb) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:48.432607 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dfd030-dece-4fd7-884d-6e96dac47b65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:48.435213 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dfd030-dece-4fd7-884d-6e96dac47b65 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:48.441906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 172/357] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Sep 1 16:17:48 2022] GET /identity/v3/role_assignments?group.id=109db92706954923b3c4932e0a7d9e5e&scope.project.id=9ea4cc94eaf84d0c8fec375d056242eb&role.id=42db5f955f2644b1bb22b0a68f8cd829 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:48.443532 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02404cac-ddce-4a3a-a3df-aff6617c3d2b None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:48.445122 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02404cac-ddce-4a3a-a3df-aff6617c3d2b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:48.447035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02404cac-ddce-4a3a-a3df-aff6617c3d2b None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:48.449798 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 173/358] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:48.454255 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02404cac-ddce-4a3a-a3df-aff6617c3d2b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:48.458336 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31318461-3321-45a6-9226-74cd594a11a3 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:48.509686 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6be1d-d2dd-4ca3-b6be-97f804bcbf4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:48.511171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6be1d-d2dd-4ca3-b6be-97f804bcbf4d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:48.512532 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6be1d-d2dd-4ca3-b6be-97f804bcbf4d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:48.513033 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6be1d-d2dd-4ca3-b6be-97f804bcbf4d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:48.527739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb6be1d-d2dd-4ca3-b6be-97f804bcbf4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:48.535095 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-3bb6be1d-d2dd-4ca3-b6be-97f804bcbf4d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:17:48.535095 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:48.535095 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:48.535095 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:48.535095 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:48.535095 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:48.535095 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:48.535095 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:48.535095 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:48.535095 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:48.535095 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:48.535095 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:48.535095 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:48.535095 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:48.535095 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:48.535095 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:48.536094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:48.536094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:48.536094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:48.536094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:48.536094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:48.536094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:48.536094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:48.536094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:48.536094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:48.536094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:48.536094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:48.536094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:48.536094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:48.536094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:48.536094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:48.536094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:48.536901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:48.536901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:48.536901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:48.536901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:48.536901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:48.536901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:48.536901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:48.536901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:48.536901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:48.536901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:48.536901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:17:48.536901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:17:48.539755 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 186/359] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:48.540900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6be1d-d2dd-4ca3-b6be-97f804bcbf4d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:48.567371 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:48.603965 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02404cac-ddce-4a3a-a3df-aff6617c3d2b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:48.607657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:48.651328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a770c7-81bb-44c4-80ef-9955c66445ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:48.651976 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a770c7-81bb-44c4-80ef-9955c66445ca None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:48.652234 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a770c7-81bb-44c4-80ef-9955c66445ca None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:48.652511 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a770c7-81bb-44c4-80ef-9955c66445ca None admin] PATH_INFO: `/v3/groups/109db92706954923b3c4932e0a7d9e5e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:48.653636 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a770c7-81bb-44c4-80ef-9955c66445ca None admin] RBAC: Authorizing `identity:get_group(group_id=109db92706954923b3c4932e0a7d9e5e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:48.655134 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a770c7-81bb-44c4-80ef-9955c66445ca None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:48.656860 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 174/360] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Sep 1 16:17:48 2022] GET /identity/v3/groups/109db92706954923b3c4932e0a7d9e5e => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:48.662811 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142431af-e47c-49e4-a3d9-ee7e70c6b6df None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:48.663053 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142431af-e47c-49e4-a3d9-ee7e70c6b6df None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:48.663781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142431af-e47c-49e4-a3d9-ee7e70c6b6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:48.675985 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6be1d-d2dd-4ca3-b6be-97f804bcbf4d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:48.691006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-142431af-e47c-49e4-a3d9-ee7e70c6b6df None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:48.692846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-142431af-e47c-49e4-a3d9-ee7e70c6b6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:48.731664 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ede85-fb65-418a-836a-0c3c95755966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:48.732308 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ede85-fb65-418a-836a-0c3c95755966 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:48.733533 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ede85-fb65-418a-836a-0c3c95755966 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:48.733887 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ede85-fb65-418a-836a-0c3c95755966 None admin] PATH_INFO: `/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:48.735103 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ede85-fb65-418a-836a-0c3c95755966 None admin] RBAC: Authorizing `identity:get_project(project_id=9ea4cc94eaf84d0c8fec375d056242eb)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:48.736553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ede85-fb65-418a-836a-0c3c95755966 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:48.738179 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 187/361] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:48 2022] GET /identity/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:48.739032 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ede85-fb65-418a-836a-0c3c95755966 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:48.784591 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 175/362] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:48.785638 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142431af-e47c-49e4-a3d9-ee7e70c6b6df None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:48.871436 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:48.886818 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ede85-fb65-418a-836a-0c3c95755966 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:48.917190 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:48.921035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142431af-e47c-49e4-a3d9-ee7e70c6b6df None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:48.924463 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db19463-d0a8-49af-ba97-d932f210c34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:48.924917 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db19463-d0a8-49af-ba97-d932f210c34d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:48.925202 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db19463-d0a8-49af-ba97-d932f210c34d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:48.925477 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db19463-d0a8-49af-ba97-d932f210c34d None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:48.926158 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db19463-d0a8-49af-ba97-d932f210c34d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:48.926557 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db19463-d0a8-49af-ba97-d932f210c34d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:48.927936 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db19463-d0a8-49af-ba97-d932f210c34d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:48.937243 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 188/363] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Sep 1 16:17:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:48.938094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db19463-d0a8-49af-ba97-d932f210c34d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:48.959740 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25123e-3d7c-4c32-bda5-678c7adec0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:48.960625 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25123e-3d7c-4c32-bda5-678c7adec0b1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:48.961048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25123e-3d7c-4c32-bda5-678c7adec0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:48.961463 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25123e-3d7c-4c32-bda5-678c7adec0b1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:48.973364 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25123e-3d7c-4c32-bda5-678c7adec0b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:48.979034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-1d25123e-3d7c-4c32-bda5-678c7adec0b1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:17:48.979034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:48.979034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:48.979034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:48.979034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:48.979034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:48.979034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:48.979034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:48.979034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:48.979034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:48.979034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:48.979034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:48.979034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:48.979034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:48.979034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:48.979034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:48.980140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:48.980140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:48.980140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:48.980140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:48.980140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:48.980140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:48.980140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:48.980140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:48.980140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:48.980140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:48.980140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:48.980140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:48.980140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:48.980140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:48.980140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:48.980140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:48.981118 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:48.981118 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:48.981118 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:48.981118 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:48.981118 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:48.981118 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:48.981118 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:48.981118 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:48.981118 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:48.981118 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:48.981118 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:17:48.981118 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:17:48.983299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 176/364] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Sep 1 16:17:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:48.984241 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25123e-3d7c-4c32-bda5-678c7adec0b1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:49.055093 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db19463-d0a8-49af-ba97-d932f210c34d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:49.091834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7bd318-cf8b-48e4-aef3-4274addb3bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:49.092445 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bd318-cf8b-48e4-aef3-4274addb3bbe None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:49.092727 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bd318-cf8b-48e4-aef3-4274addb3bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:49.093094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bd318-cf8b-48e4-aef3-4274addb3bbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:49.093406 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25123e-3d7c-4c32-bda5-678c7adec0b1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:49.093843 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7bd318-cf8b-48e4-aef3-4274addb3bbe None admin] RBAC: Adding query filter params (group.id=109db92706954923b3c4932e0a7d9e5e, role.id=42db5f955f2644b1bb22b0a68f8cd829, scope.project.id=2375c71530404fc591cb968d121ae21e) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:49.094220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7bd318-cf8b-48e4-aef3-4274addb3bbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:49.095675 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7bd318-cf8b-48e4-aef3-4274addb3bbe None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:49.101467 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 189/365] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Sep 1 16:17:48 2022] GET /identity/v3/role_assignments?group.id=109db92706954923b3c4932e0a7d9e5e&scope.project.id=2375c71530404fc591cb968d121ae21e&role.id=42db5f955f2644b1bb22b0a68f8cd829 => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:49.106221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7bd318-cf8b-48e4-aef3-4274addb3bbe None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:49.119805 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:49.159903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6952aeb-8c7e-45e3-b317-5f8734bca161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:49.160632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6952aeb-8c7e-45e3-b317-5f8734bca161 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:49.160917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6952aeb-8c7e-45e3-b317-5f8734bca161 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:49.161194 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6952aeb-8c7e-45e3-b317-5f8734bca161 None admin] PATH_INFO: `/v3/groups/109db92706954923b3c4932e0a7d9e5e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:49.162474 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6952aeb-8c7e-45e3-b317-5f8734bca161 None admin] RBAC: Authorizing `identity:get_group(group_id=109db92706954923b3c4932e0a7d9e5e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:49.164713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6952aeb-8c7e-45e3-b317-5f8734bca161 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:49.170311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 177/366] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Sep 1 16:17:48 2022] GET /identity/v3/groups/109db92706954923b3c4932e0a7d9e5e => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:49.171594 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6952aeb-8c7e-45e3-b317-5f8734bca161 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:49.211472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:49.249594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7bd318-cf8b-48e4-aef3-4274addb3bbe None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:49.291354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb25676-884f-4a26-b9f5-fd7d33be2871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:49.292276 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb25676-884f-4a26-b9f5-fd7d33be2871 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:49.292566 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb25676-884f-4a26-b9f5-fd7d33be2871 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:49.292843 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb25676-884f-4a26-b9f5-fd7d33be2871 None admin] PATH_INFO: `/v3/users/d3f84838faef4437895581a402fa2e29` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:49.294094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb25676-884f-4a26-b9f5-fd7d33be2871 None admin] RBAC: Authorizing `identity:get_user(user_id=d3f84838faef4437895581a402fa2e29)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:49.294406 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6952aeb-8c7e-45e3-b317-5f8734bca161 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:49.295582 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb25676-884f-4a26-b9f5-fd7d33be2871 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:49.297788 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 190/367] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:17:49 2022] GET /identity/v3/users/d3f84838faef4437895581a402fa2e29 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:49.298148 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb25676-884f-4a26-b9f5-fd7d33be2871 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:49.333407 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d39ae-021d-4e26-93bd-f7c6c61bc671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:49.334344 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d39ae-021d-4e26-93bd-f7c6c61bc671 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:49.334778 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d39ae-021d-4e26-93bd-f7c6c61bc671 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:49.335058 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d39ae-021d-4e26-93bd-f7c6c61bc671 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:49.335742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d39ae-021d-4e26-93bd-f7c6c61bc671 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:49.336039 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d39ae-021d-4e26-93bd-f7c6c61bc671 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:49.339068 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d39ae-021d-4e26-93bd-f7c6c61bc671 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:49.348771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 178/368] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:17:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:49.349712 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d39ae-021d-4e26-93bd-f7c6c61bc671 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:49.414886 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:49.441861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb25676-884f-4a26-b9f5-fd7d33be2871 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:49.462576 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:49.477250 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d39ae-021d-4e26-93bd-f7c6c61bc671 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:49.488117 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0896247-62b0-426f-9bd4-6f8201fdb556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:49.489166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0896247-62b0-426f-9bd4-6f8201fdb556 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:49.489447 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0896247-62b0-426f-9bd4-6f8201fdb556 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:49.490002 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0896247-62b0-426f-9bd4-6f8201fdb556 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:49.503488 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0896247-62b0-426f-9bd4-6f8201fdb556 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:49.511611 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-d0896247-62b0-426f-9bd4-6f8201fdb556 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:17:49.511611 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:49.511611 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:49.511611 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:49.511611 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:49.511611 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:49.511611 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:49.511611 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:49.511611 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:49.511611 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:49.511611 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:49.511611 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:49.511611 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:49.511611 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:49.511611 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:49.511611 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:49.512961 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:49.512961 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:49.512961 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:49.512961 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:49.512961 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:49.512961 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:49.512961 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:49.512961 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:49.512961 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:49.512961 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:49.512961 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:49.512961 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:49.512961 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:49.512961 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:49.512961 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:49.512961 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:49.514151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:49.514151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:49.514151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:49.514151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:49.514151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:49.514151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:49.514151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:49.514151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:49.514151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:49.514151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:49.514151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:17:49.514151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:17:49.514151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 191/369] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:17:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:49.515142 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0896247-62b0-426f-9bd4-6f8201fdb556 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:49.524166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cfc1d6-812c-4d8c-b329-23d210f56efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:49.524822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfc1d6-812c-4d8c-b329-23d210f56efd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:49.525104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfc1d6-812c-4d8c-b329-23d210f56efd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:49.525381 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfc1d6-812c-4d8c-b329-23d210f56efd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:49.526117 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cfc1d6-812c-4d8c-b329-23d210f56efd None admin] RBAC: Adding query filter params (group.id=109db92706954923b3c4932e0a7d9e5e, role.id=72700efc336a4b4f992b08ff9ea99f0c, scope.project.id=9ea4cc94eaf84d0c8fec375d056242eb) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:49.526584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cfc1d6-812c-4d8c-b329-23d210f56efd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:49.528186 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cfc1d6-812c-4d8c-b329-23d210f56efd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:49.535161 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 179/370] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Sep 1 16:17:49 2022] GET /identity/v3/role_assignments?group.id=109db92706954923b3c4932e0a7d9e5e&scope.project.id=9ea4cc94eaf84d0c8fec375d056242eb&role.id=72700efc336a4b4f992b08ff9ea99f0c => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:49.536068 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cfc1d6-812c-4d8c-b329-23d210f56efd None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:49.688956 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0896247-62b0-426f-9bd4-6f8201fdb556 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:49.708220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cfc1d6-812c-4d8c-b329-23d210f56efd None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:49.749924 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f5e6e-224a-479d-96f5-a22bd4248afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:49.751637 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f5e6e-224a-479d-96f5-a22bd4248afe None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:49.752147 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f5e6e-224a-479d-96f5-a22bd4248afe None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:49.752831 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f5e6e-224a-479d-96f5-a22bd4248afe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:49.754445 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f5e6e-224a-479d-96f5-a22bd4248afe None admin] RBAC: Adding query filter params (role.id=d657364ebb9c4110bc1acfe8a9f2f3fd, user.id=d3f84838faef4437895581a402fa2e29, scope.system=all) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:49.755505 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f5e6e-224a-479d-96f5-a22bd4248afe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:49.758426 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f5e6e-224a-479d-96f5-a22bd4248afe None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:49.772143 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:49.785085 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 192/371] 10.222.0.58 () {56 vars in 1308 bytes} [Thu Sep 1 16:17:49 2022] GET /identity/v3/role_assignments?user.id=d3f84838faef4437895581a402fa2e29&scope.system=all&role.id=d657364ebb9c4110bc1acfe8a9f2f3fd => generated 510 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:49.786362 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f5e6e-224a-479d-96f5-a22bd4248afe None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:49.794608 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b013abd2-1b54-4477-a1ce-577c40aecc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:49.795612 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013abd2-1b54-4477-a1ce-577c40aecc6c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:49.798873 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013abd2-1b54-4477-a1ce-577c40aecc6c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:49.799437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013abd2-1b54-4477-a1ce-577c40aecc6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:49.849207 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:49.904697 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b013abd2-1b54-4477-a1ce-577c40aecc6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:49.909004 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b013abd2-1b54-4477-a1ce-577c40aecc6c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:50.025865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 180/372] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:17:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:17:50.031275 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b013abd2-1b54-4477-a1ce-577c40aecc6c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:50.086341 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f5e6e-224a-479d-96f5-a22bd4248afe None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:50.097323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:50.147675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b013abd2-1b54-4477-a1ce-577c40aecc6c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:50.191371 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ea93f-9b9e-4c49-b8f5-95e8a14142f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:50.192220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ea93f-9b9e-4c49-b8f5-95e8a14142f3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:50.192624 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ea93f-9b9e-4c49-b8f5-95e8a14142f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:50.193000 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ea93f-9b9e-4c49-b8f5-95e8a14142f3 None admin] PATH_INFO: `/v3/projects/2375c71530404fc591cb968d121ae21e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:50.194298 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ea93f-9b9e-4c49-b8f5-95e8a14142f3 None admin] RBAC: Authorizing `identity:get_project(project_id=2375c71530404fc591cb968d121ae21e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:50.195907 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ea93f-9b9e-4c49-b8f5-95e8a14142f3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:50.198076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 181/373] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:50 2022] GET /identity/v3/projects/2375c71530404fc591cb968d121ae21e => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:50.204192 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ea93f-9b9e-4c49-b8f5-95e8a14142f3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:50.207887 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6481100-cdf5-40e1-8ae2-360294944d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:50.208517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6481100-cdf5-40e1-8ae2-360294944d6e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:50.208797 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6481100-cdf5-40e1-8ae2-360294944d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:50.209074 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6481100-cdf5-40e1-8ae2-360294944d6e None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:50.209755 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6481100-cdf5-40e1-8ae2-360294944d6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:50.210061 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6481100-cdf5-40e1-8ae2-360294944d6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:50.212947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6481100-cdf5-40e1-8ae2-360294944d6e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:50.240434 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 193/374] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 454 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:50.247194 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6481100-cdf5-40e1-8ae2-360294944d6e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:50.312519 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ea93f-9b9e-4c49-b8f5-95e8a14142f3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:50.345011 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:50.355004 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9729d36-d817-451b-89de-2aa528623b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:50.355652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9729d36-d817-451b-89de-2aa528623b07 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:50.355978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9729d36-d817-451b-89de-2aa528623b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:50.357301 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9729d36-d817-451b-89de-2aa528623b07 None admin] PATH_INFO: `/v3/groups/109db92706954923b3c4932e0a7d9e5e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:50.359206 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9729d36-d817-451b-89de-2aa528623b07 None admin] RBAC: Authorizing `identity:get_group(group_id=109db92706954923b3c4932e0a7d9e5e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:50.360684 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9729d36-d817-451b-89de-2aa528623b07 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:50.366623 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 182/375] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Sep 1 16:17:50 2022] GET /identity/v3/groups/109db92706954923b3c4932e0a7d9e5e => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:50.372803 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9729d36-d817-451b-89de-2aa528623b07 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:50.423404 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:50.490615 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9729d36-d817-451b-89de-2aa528623b07 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:50.535209 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89a465-49fa-4395-b259-53de0351542e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:50.535984 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89a465-49fa-4395-b259-53de0351542e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:50.536276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89a465-49fa-4395-b259-53de0351542e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:50.536556 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89a465-49fa-4395-b259-53de0351542e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:50.537294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d89a465-49fa-4395-b259-53de0351542e None admin] RBAC: Adding query filter params (group.id=109db92706954923b3c4932e0a7d9e5e, role.id=72700efc336a4b4f992b08ff9ea99f0c, scope.project.id=2375c71530404fc591cb968d121ae21e) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:50.537609 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d89a465-49fa-4395-b259-53de0351542e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:50.539527 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d89a465-49fa-4395-b259-53de0351542e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:50.546263 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 183/376] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Sep 1 16:17:50 2022] GET /identity/v3/role_assignments?group.id=109db92706954923b3c4932e0a7d9e5e&scope.project.id=2375c71530404fc591cb968d121ae21e&role.id=72700efc336a4b4f992b08ff9ea99f0c => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:50.552406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6481100-cdf5-40e1-8ae2-360294944d6e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:50.625688 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea0b271-9e91-430e-beca-42ace43c1a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:50.626271 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0b271-9e91-430e-beca-42ace43c1a0e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:50.626558 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0b271-9e91-430e-beca-42ace43c1a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:50.626834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0b271-9e91-430e-beca-42ace43c1a0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:50.729522 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea0b271-9e91-430e-beca-42ace43c1a0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:50.730999 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea0b271-9e91-430e-beca-42ace43c1a0e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:50.797697 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:50.851052 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 194/377] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:17:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:17:50.857451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89a465-49fa-4395-b259-53de0351542e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:50.870240 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90885c-381d-45f2-b7c0-5f008f16db04 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:50.870240 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90885c-381d-45f2-b7c0-5f008f16db04 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:50.870502 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90885c-381d-45f2-b7c0-5f008f16db04 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:50.872092 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 195/378] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:17:50.877985 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08daa8e-9213-434f-a631-db95773f6bb1 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:50.878183 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08daa8e-9213-434f-a631-db95773f6bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:50.878509 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08daa8e-9213-434f-a631-db95773f6bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:50.910117 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f08daa8e-9213-434f-a631-db95773f6bb1 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:50.912255 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f08daa8e-9213-434f-a631-db95773f6bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:50.961776 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:51.041647 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89a465-49fa-4395-b259-53de0351542e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:51.044627 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 196/379] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:51.081760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be859f49-1cbf-4efe-9813-b3b8ef008ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:51.082423 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be859f49-1cbf-4efe-9813-b3b8ef008ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:51.082707 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be859f49-1cbf-4efe-9813-b3b8ef008ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:51.082979 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be859f49-1cbf-4efe-9813-b3b8ef008ea4 None admin] PATH_INFO: `/v3/projects/e2c38221adde4053b2b37201b4051903` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:51.084107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be859f49-1cbf-4efe-9813-b3b8ef008ea4 None admin] RBAC: Authorizing `identity:get_project(project_id=e2c38221adde4053b2b37201b4051903)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:51.085612 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be859f49-1cbf-4efe-9813-b3b8ef008ea4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:51.088166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 184/380] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:50 2022] GET /identity/v3/projects/e2c38221adde4053b2b37201b4051903 => generated 344 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:51.093714 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08daa8e-9213-434f-a631-db95773f6bb1 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:51.202520 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:51.214405 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08daa8e-9213-434f-a631-db95773f6bb1 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:51.258077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b440af6-f639-495b-a12e-be6fb4ca998d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:51.259250 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b440af6-f639-495b-a12e-be6fb4ca998d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:51.259534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b440af6-f639-495b-a12e-be6fb4ca998d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:51.259805 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b440af6-f639-495b-a12e-be6fb4ca998d None admin] PATH_INFO: `/v3/groups/e473cd65b6624884a56c7e28e2d8ebb9` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:51.270888 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b440af6-f639-495b-a12e-be6fb4ca998d None admin] RBAC: Authorizing `identity:get_group(group_id=e473cd65b6624884a56c7e28e2d8ebb9)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:51.272323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b440af6-f639-495b-a12e-be6fb4ca998d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:51.274242 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 197/381] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Sep 1 16:17:51 2022] GET /identity/v3/groups/e473cd65b6624884a56c7e28e2d8ebb9 => generated 229 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:51.280678 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be859f49-1cbf-4efe-9813-b3b8ef008ea4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:51.397922 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be859f49-1cbf-4efe-9813-b3b8ef008ea4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:51.423008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:51.440087 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ee524-cc87-4df8-8a2c-98ba430b3105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:51.440692 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ee524-cc87-4df8-8a2c-98ba430b3105 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:51.440973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ee524-cc87-4df8-8a2c-98ba430b3105 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:51.441250 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ee524-cc87-4df8-8a2c-98ba430b3105 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:51.441985 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ee524-cc87-4df8-8a2c-98ba430b3105 None admin] RBAC: Adding query filter params (group.id=e473cd65b6624884a56c7e28e2d8ebb9, role.id=ff293acb8551463aafe6970230042b89, scope.project.id=e2c38221adde4053b2b37201b4051903) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:51.442326 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ee524-cc87-4df8-8a2c-98ba430b3105 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:51.443775 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ee524-cc87-4df8-8a2c-98ba430b3105 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:51.452030 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 185/382] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Sep 1 16:17:51 2022] GET /identity/v3/role_assignments?group.id=e473cd65b6624884a56c7e28e2d8ebb9&scope.project.id=e2c38221adde4053b2b37201b4051903&role.id=ff293acb8551463aafe6970230042b89 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:51.507759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b440af6-f639-495b-a12e-be6fb4ca998d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:51.581178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb3615-ae90-480c-89d5-7413980d69bb None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:51.581818 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb3615-ae90-480c-89d5-7413980d69bb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:51.582340 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb3615-ae90-480c-89d5-7413980d69bb None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:51.584142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 186/383] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:51.590849 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfb968-39f4-4444-87ee-b301852cd051 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:51.591430 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfb968-39f4-4444-87ee-b301852cd051 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:51.591931 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfb968-39f4-4444-87ee-b301852cd051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:51.619715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4edfb968-39f4-4444-87ee-b301852cd051 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:51.621045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4edfb968-39f4-4444-87ee-b301852cd051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:51.683102 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b440af6-f639-495b-a12e-be6fb4ca998d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:51.714181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 187/384] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:17:51.749231 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895573c-8927-4a43-9d4f-58d0378cd02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:51.749867 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895573c-8927-4a43-9d4f-58d0378cd02a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:51.750147 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895573c-8927-4a43-9d4f-58d0378cd02a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:51.750443 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895573c-8927-4a43-9d4f-58d0378cd02a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:51.771385 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a895573c-8927-4a43-9d4f-58d0378cd02a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:51.776686 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-a895573c-8927-4a43-9d4f-58d0378cd02a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:51.776686 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:51.776686 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:51.776686 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:51.776686 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:51.776686 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:51.776686 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:51.776686 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:51.776686 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:51.776686 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:51.776686 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:51.776686 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:51.776686 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:51.776686 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:51.776686 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:51.776686 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:51.777624 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:51.777624 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:51.777624 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:51.777624 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:51.777624 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:51.777624 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:51.777624 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:51.777624 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:51.777624 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:51.777624 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:51.777624 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:51.777624 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:51.777624 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:51.777624 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:51.777624 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:51.777624 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:51.778578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:51.778578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:51.778578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:51.778578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:51.778578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:51.778578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:51.778578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:51.778578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:51.778578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:51.778578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:51.778578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:51.778578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:17:51.778578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 198/385] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:51.783895 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edfb968-39f4-4444-87ee-b301852cd051 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:51.950115 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edfb968-39f4-4444-87ee-b301852cd051 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:51.982411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:52.019847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f20d2d-ba9f-4ec4-931e-dcc1fd1926fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:52.020484 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f20d2d-ba9f-4ec4-931e-dcc1fd1926fc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:52.020771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f20d2d-ba9f-4ec4-931e-dcc1fd1926fc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:52.021046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f20d2d-ba9f-4ec4-931e-dcc1fd1926fc None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:52.021777 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f20d2d-ba9f-4ec4-931e-dcc1fd1926fc None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:52.022028 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f20d2d-ba9f-4ec4-931e-dcc1fd1926fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:52.025046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f20d2d-ba9f-4ec4-931e-dcc1fd1926fc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:52.039985 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 188/386] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Sep 1 16:17:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:52.047748 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895573c-8927-4a43-9d4f-58d0378cd02a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:52.082511 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:52.087849 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f20d2d-ba9f-4ec4-931e-dcc1fd1926fc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:52.194528 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895573c-8927-4a43-9d4f-58d0378cd02a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:52.222051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f20d2d-ba9f-4ec4-931e-dcc1fd1926fc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:52.239225 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea8d72-fed9-426a-a6bf-d4a3cc5c67fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:52.239883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea8d72-fed9-426a-a6bf-d4a3cc5c67fe None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:52.240166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea8d72-fed9-426a-a6bf-d4a3cc5c67fe None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:52.240446 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea8d72-fed9-426a-a6bf-d4a3cc5c67fe None admin] PATH_INFO: `/v3/groups/109db92706954923b3c4932e0a7d9e5e` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:52.241801 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ea8d72-fed9-426a-a6bf-d4a3cc5c67fe None admin] RBAC: Authorizing `identity:get_group(group_id=109db92706954923b3c4932e0a7d9e5e)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:52.243355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ea8d72-fed9-426a-a6bf-d4a3cc5c67fe None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:52.245289 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 199/387] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Sep 1 16:17:52 2022] GET /identity/v3/groups/109db92706954923b3c4932e0a7d9e5e => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:52.250655 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea8d72-fed9-426a-a6bf-d4a3cc5c67fe None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:52.268597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e51ae0-cf22-432f-8fde-ccb5d5745c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:52.269533 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e51ae0-cf22-432f-8fde-ccb5d5745c20 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:52.270026 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e51ae0-cf22-432f-8fde-ccb5d5745c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:52.270544 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e51ae0-cf22-432f-8fde-ccb5d5745c20 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:52.283961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e51ae0-cf22-432f-8fde-ccb5d5745c20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:52.290698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-a7e51ae0-cf22-432f-8fde-ccb5d5745c20 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:52.290698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:52.290698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:52.290698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:52.290698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:52.290698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:52.290698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:52.290698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:52.290698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:52.290698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:52.290698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:52.290698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:52.290698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:52.290698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:52.290698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:52.290698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:52.291739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:52.291739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:52.291739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:52.291739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:52.291739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:52.291739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:52.291739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:52.291739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:52.291739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:52.291739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:52.291739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:52.291739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:52.291739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:52.291739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:52.291739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:52.291739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:52.292846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:52.292846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:52.292846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:52.292846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:52.292846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:52.292846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:52.292846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:52.292846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:52.292846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:52.292846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:52.292846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:52.292846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:17:52.299847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 189/388] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:52.304099 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e51ae0-cf22-432f-8fde-ccb5d5745c20 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:52.360943 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:52.366648 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:52.385369 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea8d72-fed9-426a-a6bf-d4a3cc5c67fe None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:52.426480 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee91348-1f4d-4ff1-934f-a4607505930d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:52.429029 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee91348-1f4d-4ff1-934f-a4607505930d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:52.429465 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee91348-1f4d-4ff1-934f-a4607505930d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:52.429970 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee91348-1f4d-4ff1-934f-a4607505930d None admin] PATH_INFO: `/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:52.431910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee91348-1f4d-4ff1-934f-a4607505930d None admin] RBAC: Authorizing `identity:get_project(project_id=9ea4cc94eaf84d0c8fec375d056242eb)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:52.433640 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee91348-1f4d-4ff1-934f-a4607505930d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:52.435905 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 200/389] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:52 2022] GET /identity/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:52.442226 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee91348-1f4d-4ff1-934f-a4607505930d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:52.445858 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e51ae0-cf22-432f-8fde-ccb5d5745c20 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:52.487020 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8cd22-e851-4d79-b45a-edc27ed45faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:52.487855 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8cd22-e851-4d79-b45a-edc27ed45faa None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:52.488216 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8cd22-e851-4d79-b45a-edc27ed45faa None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:52.488593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8cd22-e851-4d79-b45a-edc27ed45faa None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:52.489376 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8cd22-e851-4d79-b45a-edc27ed45faa None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:52.489906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8cd22-e851-4d79-b45a-edc27ed45faa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:52.491515 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8cd22-e851-4d79-b45a-edc27ed45faa None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:52.506218 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 190/390] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Sep 1 16:17:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:52.507077 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302c1b9-c19d-4417-95b2-1bcecce15dc1 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:52.508474 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302c1b9-c19d-4417-95b2-1bcecce15dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:52.508474 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302c1b9-c19d-4417-95b2-1bcecce15dc1 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:52.511882 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 191/391] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:52.511882 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2302c1b9-c19d-4417-95b2-1bcecce15dc1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:52.589250 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee91348-1f4d-4ff1-934f-a4607505930d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:52.631335 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec174e8-bde6-4e2d-ae2d-e7989e6d87fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:52.632184 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec174e8-bde6-4e2d-ae2d-e7989e6d87fb None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:52.632470 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec174e8-bde6-4e2d-ae2d-e7989e6d87fb None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:52.632745 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec174e8-bde6-4e2d-ae2d-e7989e6d87fb None admin] PATH_INFO: `/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb/groups/109db92706954923b3c4932e0a7d9e5e/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:52.635927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec174e8-bde6-4e2d-ae2d-e7989e6d87fb None admin] RBAC: Authorizing `identity:create_grant(project_id=9ea4cc94eaf84d0c8fec375d056242eb, group_id=109db92706954923b3c4932e0a7d9e5e, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:52.638601 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec174e8-bde6-4e2d-ae2d-e7989e6d87fb None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:52.639141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:52.643756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2302c1b9-c19d-4417-95b2-1bcecce15dc1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:52.657601 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 201/392] 10.222.0.58 () {58 vars in 1442 bytes} [Thu Sep 1 16:17:52 2022] PUT /identity/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb/groups/109db92706954923b3c4932e0a7d9e5e/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:17:52.660580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f315a61-4fe6-46b8-baa5-fd724da97ee1 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:52.661515 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f315a61-4fe6-46b8-baa5-fd724da97ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:52.662005 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f315a61-4fe6-46b8-baa5-fd724da97ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:52.695952 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e377254-245b-491a-ab82-a02ea2df49ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:52.697624 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e377254-245b-491a-ab82-a02ea2df49ee None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:52.697885 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e377254-245b-491a-ab82-a02ea2df49ee None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:52.698041 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-4f315a61-4fe6-46b8-baa5-fd724da97ee1 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:52.698579 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e377254-245b-491a-ab82-a02ea2df49ee None admin] PATH_INFO: `/v3/groups/109db92706954923b3c4932e0a7d9e5e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:52.699577 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e377254-245b-491a-ab82-a02ea2df49ee None admin] RBAC: Authorizing `identity:get_group(group_id=109db92706954923b3c4932e0a7d9e5e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:52.700090 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4f315a61-4fe6-46b8-baa5-fd724da97ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:52.701251 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e377254-245b-491a-ab82-a02ea2df49ee None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:52.703588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 192/393] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Sep 1 16:17:52 2022] GET /identity/v3/groups/109db92706954923b3c4932e0a7d9e5e => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:52.711534 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e377254-245b-491a-ab82-a02ea2df49ee None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:52.712638 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:52.787803 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 202/394] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:52.840501 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e377254-245b-491a-ab82-a02ea2df49ee None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:52.877560 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c117a9-2776-449a-96c2-d717beab3708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:52.878238 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c117a9-2776-449a-96c2-d717beab3708 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:52.878528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c117a9-2776-449a-96c2-d717beab3708 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:52.878969 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c117a9-2776-449a-96c2-d717beab3708 None admin] PATH_INFO: `/v3/projects/2375c71530404fc591cb968d121ae21e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:52.880207 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c117a9-2776-449a-96c2-d717beab3708 None admin] RBAC: Authorizing `identity:get_project(project_id=2375c71530404fc591cb968d121ae21e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:52.881758 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c117a9-2776-449a-96c2-d717beab3708 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:52.883986 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 193/395] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:52 2022] GET /identity/v3/projects/2375c71530404fc591cb968d121ae21e => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:52.891280 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f315a61-4fe6-46b8-baa5-fd724da97ee1 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:53.010452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f315a61-4fe6-46b8-baa5-fd724da97ee1 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:53.058281 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe742a83-1276-46ca-9874-e270da1f8481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:53.058958 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe742a83-1276-46ca-9874-e270da1f8481 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:53.059238 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe742a83-1276-46ca-9874-e270da1f8481 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:53.059513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe742a83-1276-46ca-9874-e270da1f8481 None admin] PATH_INFO: `/v3/projects/2375c71530404fc591cb968d121ae21e/groups/109db92706954923b3c4932e0a7d9e5e/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:53.065243 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe742a83-1276-46ca-9874-e270da1f8481 None admin] RBAC: Authorizing `identity:create_grant(project_id=2375c71530404fc591cb968d121ae21e, group_id=109db92706954923b3c4932e0a7d9e5e, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:53.066445 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe742a83-1276-46ca-9874-e270da1f8481 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:53.077626 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:53.084893 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 203/396] 10.222.0.58 () {58 vars in 1442 bytes} [Thu Sep 1 16:17:52 2022] PUT /identity/v3/projects/2375c71530404fc591cb968d121ae21e/groups/109db92706954923b3c4932e0a7d9e5e/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:17:53.120717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c117a9-2776-449a-96c2-d717beab3708 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:53.168800 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:53.245405 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c117a9-2776-449a-96c2-d717beab3708 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:53.282947 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c56e46-d7f7-4213-bd72-446e6cf49739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:53.283698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c56e46-d7f7-4213-bd72-446e6cf49739 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:53.283988 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c56e46-d7f7-4213-bd72-446e6cf49739 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:53.284269 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c56e46-d7f7-4213-bd72-446e6cf49739 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:53.296155 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c56e46-d7f7-4213-bd72-446e6cf49739 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:53.301739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-71c56e46-d7f7-4213-bd72-446e6cf49739 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:17:53.301739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:53.301739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:53.301739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:53.301739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:53.301739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:53.301739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:53.301739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:53.301739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:53.301739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:53.301739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:53.301739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:53.301739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:53.301739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:53.301739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:53.301739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:53.302815 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:53.302815 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:53.302815 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:53.302815 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:53.302815 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:53.302815 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:53.302815 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:53.302815 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:53.302815 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:53.302815 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:53.302815 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:53.302815 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:53.302815 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:53.302815 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:53.302815 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:53.302815 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:53.303640 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:53.303640 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:53.303640 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:53.303640 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:53.303640 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:53.303640 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:53.303640 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:53.303640 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:53.303640 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:53.303640 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:53.303640 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:17:53.303640 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:17:53.308549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 194/397] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Sep 1 16:17:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:17:53.309999 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe742a83-1276-46ca-9874-e270da1f8481 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:53.342076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97832d6-38e0-4a69-81c3-0ec1b955ca3c None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:53.342762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97832d6-38e0-4a69-81c3-0ec1b955ca3c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:53.343240 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97832d6-38e0-4a69-81c3-0ec1b955ca3c None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:53.345186 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 195/398] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:53.351600 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db23c1e-8559-4761-af98-4fd79190306e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:53.351886 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db23c1e-8559-4761-af98-4fd79190306e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:53.352162 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db23c1e-8559-4761-af98-4fd79190306e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:53.377839 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8db23c1e-8559-4761-af98-4fd79190306e None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:53.379744 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8db23c1e-8559-4761-af98-4fd79190306e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:53.445110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe742a83-1276-46ca-9874-e270da1f8481 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:53.491729 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 196/399] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:53.502888 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2624ceb-b353-4dda-9e7d-cd9e766b63fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:53.503788 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2624ceb-b353-4dda-9e7d-cd9e766b63fc None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:53.504267 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2624ceb-b353-4dda-9e7d-cd9e766b63fc None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:53.504751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2624ceb-b353-4dda-9e7d-cd9e766b63fc None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:53.505629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2624ceb-b353-4dda-9e7d-cd9e766b63fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:53.506128 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2624ceb-b353-4dda-9e7d-cd9e766b63fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:53.507778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2624ceb-b353-4dda-9e7d-cd9e766b63fc None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:53.523246 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 204/400] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:17:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:53.527597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db23c1e-8559-4761-af98-4fd79190306e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:53.563708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:53.657863 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db23c1e-8559-4761-af98-4fd79190306e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:53.704230 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d49962-0f5e-4452-9323-50eb4115fe17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:53.704863 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d49962-0f5e-4452-9323-50eb4115fe17 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:53.705138 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d49962-0f5e-4452-9323-50eb4115fe17 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:53.705409 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d49962-0f5e-4452-9323-50eb4115fe17 None admin] PATH_INFO: `/v3/groups/109db92706954923b3c4932e0a7d9e5e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:53.706544 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d49962-0f5e-4452-9323-50eb4115fe17 None admin] RBAC: Authorizing `identity:get_group(group_id=109db92706954923b3c4932e0a7d9e5e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:53.707978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d49962-0f5e-4452-9323-50eb4115fe17 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:53.709826 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 197/401] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Sep 1 16:17:53 2022] GET /identity/v3/groups/109db92706954923b3c4932e0a7d9e5e => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:53.715270 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2624ceb-b353-4dda-9e7d-cd9e766b63fc None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:53.816148 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:53.858490 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d49962-0f5e-4452-9323-50eb4115fe17 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:53.860674 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2624ceb-b353-4dda-9e7d-cd9e766b63fc None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:53.923678 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a34ae3-d2b6-4d37-b7b5-27443a688485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:53.925372 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a34ae3-d2b6-4d37-b7b5-27443a688485 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:53.925658 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a34ae3-d2b6-4d37-b7b5-27443a688485 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:53.926428 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a34ae3-d2b6-4d37-b7b5-27443a688485 None admin] PATH_INFO: `/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:53.928381 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a34ae3-d2b6-4d37-b7b5-27443a688485 None admin] RBAC: Authorizing `identity:get_project(project_id=9ea4cc94eaf84d0c8fec375d056242eb)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:53.929817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a34ae3-d2b6-4d37-b7b5-27443a688485 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:53.934618 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 205/402] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:53 2022] GET /identity/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:53.943203 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a34ae3-d2b6-4d37-b7b5-27443a688485 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:53.986621 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:54.004334 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d49962-0f5e-4452-9323-50eb4115fe17 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:54.057035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996565d-5a1a-4998-ac04-8b0bf9b5083b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:54.057676 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996565d-5a1a-4998-ac04-8b0bf9b5083b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:54.057957 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996565d-5a1a-4998-ac04-8b0bf9b5083b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:54.058251 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996565d-5a1a-4998-ac04-8b0bf9b5083b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:54.076608 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9996565d-5a1a-4998-ac04-8b0bf9b5083b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:54.081620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-9996565d-5a1a-4998-ac04-8b0bf9b5083b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:17:54.081620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:54.081620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:54.081620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:54.081620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:54.081620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:54.081620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:54.081620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:54.081620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:54.081620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:54.081620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:54.081620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:54.081620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:54.081620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:54.081620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:54.081620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:54.082587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:54.082587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:54.082587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:54.082587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:54.082587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:54.082587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:54.082587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:54.082587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:54.082587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:54.082587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:54.082587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:54.082587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:54.082587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:54.082587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:54.082587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:54.082587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:54.083418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:54.083418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:54.083418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:54.083418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:54.083418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:54.083418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:54.083418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:54.083418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:54.083418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:54.083418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:54.083418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:17:54.083418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:17:54.085862 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a34ae3-d2b6-4d37-b7b5-27443a688485 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:54.091239 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 198/403] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Sep 1 16:17:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:54.096600 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996565d-5a1a-4998-ac04-8b0bf9b5083b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:54.142578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:54.155992 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2726b4d-0643-4bdb-a3d7-3c76d5f47dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:54.156665 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2726b4d-0643-4bdb-a3d7-3c76d5f47dc1 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:54.156949 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2726b4d-0643-4bdb-a3d7-3c76d5f47dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:54.157227 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2726b4d-0643-4bdb-a3d7-3c76d5f47dc1 None admin] PATH_INFO: `/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb/groups/109db92706954923b3c4932e0a7d9e5e/roles/72700efc336a4b4f992b08ff9ea99f0c` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:54.163014 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2726b4d-0643-4bdb-a3d7-3c76d5f47dc1 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ea4cc94eaf84d0c8fec375d056242eb, group_id=109db92706954923b3c4932e0a7d9e5e, role_id=72700efc336a4b4f992b08ff9ea99f0c)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:54.164465 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2726b4d-0643-4bdb-a3d7-3c76d5f47dc1 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:54.201748 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 206/404] 10.222.0.58 () {58 vars in 1442 bytes} [Thu Sep 1 16:17:53 2022] PUT /identity/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb/groups/109db92706954923b3c4932e0a7d9e5e/roles/72700efc336a4b4f992b08ff9ea99f0c => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:17:54.237658 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75523b6c-89c5-4641-b5f9-8a294f10f347 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:54.237965 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75523b6c-89c5-4641-b5f9-8a294f10f347 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:54.238252 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75523b6c-89c5-4641-b5f9-8a294f10f347 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:54.239936 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 207/405] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:17:54.242225 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996565d-5a1a-4998-ac04-8b0bf9b5083b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:54.246187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b027cca-2abb-4e2e-97db-a5b37539f77d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:54.246187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b027cca-2abb-4e2e-97db-a5b37539f77d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:54.246187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b027cca-2abb-4e2e-97db-a5b37539f77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:54.265925 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-8b027cca-2abb-4e2e-97db-a5b37539f77d None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:54.267256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8b027cca-2abb-4e2e-97db-a5b37539f77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:54.306971 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ffa9b3-71ec-4bd8-bead-6cae008cc9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:54.307966 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffa9b3-71ec-4bd8-bead-6cae008cc9d8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:54.308449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffa9b3-71ec-4bd8-bead-6cae008cc9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:54.308968 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffa9b3-71ec-4bd8-bead-6cae008cc9d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:54.309976 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ffa9b3-71ec-4bd8-bead-6cae008cc9d8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:54.310633 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ffa9b3-71ec-4bd8-bead-6cae008cc9d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:54.312139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ffa9b3-71ec-4bd8-bead-6cae008cc9d8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:54.331760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 199/406] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:17:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:54.338008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ffa9b3-71ec-4bd8-bead-6cae008cc9d8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:54.354730 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 208/407] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:17:54.371281 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:54.464572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ffa9b3-71ec-4bd8-bead-6cae008cc9d8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:54.501101 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bbe8b5-756a-44b1-87ca-4ee57d7d35fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:54.502014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bbe8b5-756a-44b1-87ca-4ee57d7d35fe None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:54.502533 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bbe8b5-756a-44b1-87ca-4ee57d7d35fe None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:54.502990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bbe8b5-756a-44b1-87ca-4ee57d7d35fe None admin] PATH_INFO: `/v3/groups/109db92706954923b3c4932e0a7d9e5e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:54.504292 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bbe8b5-756a-44b1-87ca-4ee57d7d35fe None admin] RBAC: Authorizing `identity:get_group(group_id=109db92706954923b3c4932e0a7d9e5e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:54.505996 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bbe8b5-756a-44b1-87ca-4ee57d7d35fe None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:54.507757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 200/408] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Sep 1 16:17:54 2022] GET /identity/v3/groups/109db92706954923b3c4932e0a7d9e5e => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:54.515431 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b027cca-2abb-4e2e-97db-a5b37539f77d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:54.628118 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b027cca-2abb-4e2e-97db-a5b37539f77d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:54.649533 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:54.673949 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989eb61-39b5-4154-ad27-5f3a85deb44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:54.675417 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989eb61-39b5-4154-ad27-5f3a85deb44d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:54.675915 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989eb61-39b5-4154-ad27-5f3a85deb44d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:54.676435 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989eb61-39b5-4154-ad27-5f3a85deb44d None admin] PATH_INFO: `/v3/projects/2375c71530404fc591cb968d121ae21e` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:54.678751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989eb61-39b5-4154-ad27-5f3a85deb44d None admin] RBAC: Authorizing `identity:get_project(project_id=2375c71530404fc591cb968d121ae21e)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:54.680969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989eb61-39b5-4154-ad27-5f3a85deb44d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:54.682898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 209/409] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:54 2022] GET /identity/v3/projects/2375c71530404fc591cb968d121ae21e => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:54.688674 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bbe8b5-756a-44b1-87ca-4ee57d7d35fe None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:54.783674 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989eb61-39b5-4154-ad27-5f3a85deb44d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:54.787179 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:54.804609 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bbe8b5-756a-44b1-87ca-4ee57d7d35fe None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:54.840798 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc25bf-1d1b-4931-8994-aefaf5632260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:54.841783 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc25bf-1d1b-4931-8994-aefaf5632260 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:54.842285 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc25bf-1d1b-4931-8994-aefaf5632260 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:54.842745 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc25bf-1d1b-4931-8994-aefaf5632260 None admin] PATH_INFO: `/v3/projects/2375c71530404fc591cb968d121ae21e/groups/109db92706954923b3c4932e0a7d9e5e/roles/72700efc336a4b4f992b08ff9ea99f0c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:54.845131 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc25bf-1d1b-4931-8994-aefaf5632260 None admin] RBAC: Authorizing `identity:create_grant(project_id=2375c71530404fc591cb968d121ae21e, group_id=109db92706954923b3c4932e0a7d9e5e, role_id=72700efc336a4b4f992b08ff9ea99f0c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:54.846818 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc25bf-1d1b-4931-8994-aefaf5632260 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:54.866170 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 201/410] 10.222.0.58 () {58 vars in 1442 bytes} [Thu Sep 1 16:17:54 2022] PUT /identity/v3/projects/2375c71530404fc591cb968d121ae21e/groups/109db92706954923b3c4932e0a7d9e5e/roles/72700efc336a4b4f992b08ff9ea99f0c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:17:54.942475 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989eb61-39b5-4154-ad27-5f3a85deb44d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:54.980934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff330a-69a3-4243-9578-74a5346fa4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:54.981986 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff330a-69a3-4243-9578-74a5346fa4a5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:54.982524 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff330a-69a3-4243-9578-74a5346fa4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:54.983029 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff330a-69a3-4243-9578-74a5346fa4a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:54.995560 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff330a-69a3-4243-9578-74a5346fa4a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:55.001082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-24ff330a-69a3-4243-9578-74a5346fa4a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:17:55.001082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:55.001082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:55.001082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:55.001082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:55.001082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:55.001082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:55.001082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:55.001082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:55.001082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:55.001082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:55.001082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:55.001082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:55.001082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:55.001082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:55.001082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:55.002133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:55.002133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:55.002133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:55.002133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:55.002133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:55.002133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:55.002133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:55.002133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:55.002133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:55.002133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:55.002133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:55.002133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:55.002133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:55.002133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:55.002133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:55.002133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:55.003225 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:55.003225 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:55.003225 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:55.003225 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:55.003225 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:55.003225 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:55.003225 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:55.003225 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:55.003225 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:55.003225 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:55.003225 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:17:55.003225 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:17:55.005730 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 210/411] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:17:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:17:55.014793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc25bf-1d1b-4931-8994-aefaf5632260 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:55.164973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc25bf-1d1b-4931-8994-aefaf5632260 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:55.207360 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1124fd7-b3b0-4929-849a-03f898f6d7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:55.207967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1124fd7-b3b0-4929-849a-03f898f6d7e7 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:55.208245 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1124fd7-b3b0-4929-849a-03f898f6d7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:55.208522 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1124fd7-b3b0-4929-849a-03f898f6d7e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:55.209188 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1124fd7-b3b0-4929-849a-03f898f6d7e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:55.209488 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1124fd7-b3b0-4929-849a-03f898f6d7e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:55.210980 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1124fd7-b3b0-4929-849a-03f898f6d7e7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:55.224114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 202/412] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:55.229599 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff330a-69a3-4243-9578-74a5346fa4a5 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:55.337393 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:55.353294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c53c37-6094-421c-89e9-bed2160c9674 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:55.353829 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c53c37-6094-421c-89e9-bed2160c9674 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:55.353829 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c53c37-6094-421c-89e9-bed2160c9674 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:55.355451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 203/413] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:55.364753 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0ff82-74c7-48fe-9e2e-35289ec98e18 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:55.365037 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0ff82-74c7-48fe-9e2e-35289ec98e18 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:55.365312 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0ff82-74c7-48fe-9e2e-35289ec98e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:55.392821 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:55.398377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff330a-69a3-4243-9578-74a5346fa4a5 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:55.409976 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6af0ff82-74c7-48fe-9e2e-35289ec98e18 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:55.411261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6af0ff82-74c7-48fe-9e2e-35289ec98e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:55.464449 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2d778-f901-44b7-829a-9acb791b9f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:55.465083 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2d778-f901-44b7-829a-9acb791b9f93 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:55.465360 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2d778-f901-44b7-829a-9acb791b9f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:55.465633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2d778-f901-44b7-829a-9acb791b9f93 None admin] PATH_INFO: `/v3/groups/e473cd65b6624884a56c7e28e2d8ebb9` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:55.466882 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a2d778-f901-44b7-829a-9acb791b9f93 None admin] RBAC: Authorizing `identity:get_group(group_id=e473cd65b6624884a56c7e28e2d8ebb9)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:55.468328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a2d778-f901-44b7-829a-9acb791b9f93 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:55.472586 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 211/414] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Sep 1 16:17:55 2022] GET /identity/v3/groups/e473cd65b6624884a56c7e28e2d8ebb9 => generated 229 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:55.476314 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2d778-f901-44b7-829a-9acb791b9f93 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:55.534341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 204/415] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:17:55.616235 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2d778-f901-44b7-829a-9acb791b9f93 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:55.637724 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:55.662127 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb6705-fdf0-42f4-b732-2c5e040b37b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:55.662785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb6705-fdf0-42f4-b732-2c5e040b37b6 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:55.663060 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb6705-fdf0-42f4-b732-2c5e040b37b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:55.663330 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb6705-fdf0-42f4-b732-2c5e040b37b6 None admin] PATH_INFO: `/v3/projects/e2c38221adde4053b2b37201b4051903` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:55.664470 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb6705-fdf0-42f4-b732-2c5e040b37b6 None admin] RBAC: Authorizing `identity:get_project(project_id=e2c38221adde4053b2b37201b4051903)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:55.665933 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb6705-fdf0-42f4-b732-2c5e040b37b6 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:55.668125 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 212/416] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:55 2022] GET /identity/v3/projects/e2c38221adde4053b2b37201b4051903 => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:55.673911 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0ff82-74c7-48fe-9e2e-35289ec98e18 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:55.684472 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59605a-117c-4277-b408-b3687bd9d68f None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:55.684759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59605a-117c-4277-b408-b3687bd9d68f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:55.685032 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59605a-117c-4277-b408-b3687bd9d68f None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:55.689766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 213/417] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:55.694097 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba718a-6fcb-4c0d-947d-ce0e719e4f5c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:55.694404 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba718a-6fcb-4c0d-947d-ce0e719e4f5c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:55.694674 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba718a-6fcb-4c0d-947d-ce0e719e4f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:55.725852 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-37ba718a-6fcb-4c0d-947d-ce0e719e4f5c None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:55.727268 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-37ba718a-6fcb-4c0d-947d-ce0e719e4f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:55.806766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 214/418] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:55.836823 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0ff82-74c7-48fe-9e2e-35289ec98e18 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:55.873097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e75f1-9d66-4b68-9795-df74e8a369b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:55.874103 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e75f1-9d66-4b68-9795-df74e8a369b1 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:55.874638 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e75f1-9d66-4b68-9795-df74e8a369b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:55.875157 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e75f1-9d66-4b68-9795-df74e8a369b1 None admin] PATH_INFO: `/v3/projects/e2c38221adde4053b2b37201b4051903/groups/e473cd65b6624884a56c7e28e2d8ebb9/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:55.877356 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385e75f1-9d66-4b68-9795-df74e8a369b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=e2c38221adde4053b2b37201b4051903, group_id=e473cd65b6624884a56c7e28e2d8ebb9, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:55.879027 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385e75f1-9d66-4b68-9795-df74e8a369b1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:55.882276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:55.914690 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba718a-6fcb-4c0d-947d-ce0e719e4f5c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:55.916565 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 205/419] 10.222.0.58 () {58 vars in 1442 bytes} [Thu Sep 1 16:17:55 2022] PUT /identity/v3/projects/e2c38221adde4053b2b37201b4051903/groups/e473cd65b6624884a56c7e28e2d8ebb9/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:17:56.056580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba718a-6fcb-4c0d-947d-ce0e719e4f5c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:56.075320 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:56.107128 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be126f-7a17-4dac-b21a-9dfad4fb44ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:56.107847 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be126f-7a17-4dac-b21a-9dfad4fb44ef None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:56.108134 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be126f-7a17-4dac-b21a-9dfad4fb44ef None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:56.108496 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be126f-7a17-4dac-b21a-9dfad4fb44ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:56.124175 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69be126f-7a17-4dac-b21a-9dfad4fb44ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:56.128950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-69be126f-7a17-4dac-b21a-9dfad4fb44ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:56.128950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:56.128950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:56.128950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:56.128950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:56.128950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:56.128950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:56.128950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:56.128950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:56.128950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:56.128950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:56.128950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:56.128950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:56.128950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:56.128950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:56.128950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:56.129883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:56.129883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:56.129883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:56.129883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:56.129883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:56.129883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:56.129883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:56.129883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:56.129883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:56.129883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:56.129883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:56.129883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:56.129883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:56.129883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:56.129883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:56.129883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:56.130734 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:56.130734 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:56.130734 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:56.130734 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:56.130734 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:56.130734 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:56.130734 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:56.130734 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:56.130734 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:56.130734 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:56.130734 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:56.130734 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:17:56.130734 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 215/420] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:17:56.136760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e75f1-9d66-4b68-9795-df74e8a369b1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:56.142706 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be126f-7a17-4dac-b21a-9dfad4fb44ef None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:56.273768 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e75f1-9d66-4b68-9795-df74e8a369b1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:56.288833 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be126f-7a17-4dac-b21a-9dfad4fb44ef None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:56.313915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93feec9-3376-4fc8-950c-927901f84281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:56.314570 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93feec9-3376-4fc8-950c-927901f84281 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:56.314850 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93feec9-3376-4fc8-950c-927901f84281 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:56.315125 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93feec9-3376-4fc8-950c-927901f84281 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:56.315795 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93feec9-3376-4fc8-950c-927901f84281 None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:56.316208 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93feec9-3376-4fc8-950c-927901f84281 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:56.317585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93feec9-3376-4fc8-950c-927901f84281 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:56.328744 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 206/421] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Sep 1 16:17:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:56.332676 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd65871-5ee5-445f-9265-dd1727694e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:56.333525 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd65871-5ee5-445f-9265-dd1727694e75 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:56.333811 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd65871-5ee5-445f-9265-dd1727694e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:56.334086 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd65871-5ee5-445f-9265-dd1727694e75 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:56.334290 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93feec9-3376-4fc8-950c-927901f84281 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:56.350209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd65871-5ee5-445f-9265-dd1727694e75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:56.357355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-3fd65871-5ee5-445f-9265-dd1727694e75 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:56.357355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:56.357355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:56.357355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:56.357355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:56.357355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:56.357355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:56.357355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:56.357355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:56.357355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:56.357355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:56.357355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:56.357355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:56.357355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:56.357355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:56.357355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:56.358301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:56.358301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:56.358301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:56.358301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:56.358301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:56.358301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:56.358301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:56.358301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:56.358301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:56.358301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:56.358301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:56.358301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:56.358301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:56.358301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:56.358301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:56.358301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:56.359140 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:56.359140 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:56.359140 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:56.359140 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:56.359140 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:56.359140 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:56.359140 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:56.359140 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:56.359140 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:56.359140 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:56.359140 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:17:56.359140 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:17:56.362589 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 216/422] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:17:56.365109 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd65871-5ee5-445f-9265-dd1727694e75 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:56.407307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:56.462306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:56.489463 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd65871-5ee5-445f-9265-dd1727694e75 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:56.530089 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade35267-46a5-46d5-8a09-7864aa704ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:56.530742 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade35267-46a5-46d5-8a09-7864aa704ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:56.531008 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade35267-46a5-46d5-8a09-7864aa704ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:56.531444 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade35267-46a5-46d5-8a09-7864aa704ca4 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:56.532125 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade35267-46a5-46d5-8a09-7864aa704ca4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:56.532684 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade35267-46a5-46d5-8a09-7864aa704ca4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:56.534152 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade35267-46a5-46d5-8a09-7864aa704ca4 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:56.548292 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93feec9-3376-4fc8-950c-927901f84281 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:56.549793 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 217/423] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Sep 1 16:17:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:56.551269 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade35267-46a5-46d5-8a09-7864aa704ca4 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:56.586713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23af53-8e61-4377-9b86-37044536c76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:56.587329 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23af53-8e61-4377-9b86-37044536c76f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:56.587611 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23af53-8e61-4377-9b86-37044536c76f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:56.587882 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23af53-8e61-4377-9b86-37044536c76f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:56.655871 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:56.659967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23af53-8e61-4377-9b86-37044536c76f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:56.661413 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23af53-8e61-4377-9b86-37044536c76f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:56.662025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade35267-46a5-46d5-8a09-7864aa704ca4 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:56.702828 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b50983-fa9a-4cbe-a397-e249d6dd3854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:56.703622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b50983-fa9a-4cbe-a397-e249d6dd3854 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:56.703910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b50983-fa9a-4cbe-a397-e249d6dd3854 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:56.704189 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b50983-fa9a-4cbe-a397-e249d6dd3854 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:56.723373 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:56.769989 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 207/424] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:17:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 16:17:56.771460 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fd753-6225-4365-8b61-6c3119581de4 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:56.771802 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fd753-6225-4365-8b61-6c3119581de4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:56.772083 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fd753-6225-4365-8b61-6c3119581de4 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:56.773432 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 208/425] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:56.776916 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993fd753-6225-4365-8b61-6c3119581de4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:56.794762 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b50983-fa9a-4cbe-a397-e249d6dd3854 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:56.797065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b50983-fa9a-4cbe-a397-e249d6dd3854 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:56.891282 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993fd753-6225-4365-8b61-6c3119581de4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:56.905380 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:56.912684 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 218/426] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:17:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 16:17:56.913837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58606f-9305-43cc-a34f-e3525dc78175 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:56.914122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58606f-9305-43cc-a34f-e3525dc78175 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:56.914472 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58606f-9305-43cc-a34f-e3525dc78175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:56.936901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-1a58606f-9305-43cc-a34f-e3525dc78175 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:56.937546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f3b59-5638-4456-ac5a-73d6135c9d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:56.938135 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1a58606f-9305-43cc-a34f-e3525dc78175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:56.938521 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f3b59-5638-4456-ac5a-73d6135c9d24 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:56.938965 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f3b59-5638-4456-ac5a-73d6135c9d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:56.939385 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f3b59-5638-4456-ac5a-73d6135c9d24 None admin] PATH_INFO: `/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:56.941096 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f3b59-5638-4456-ac5a-73d6135c9d24 None admin] RBAC: Authorizing `identity:get_project(project_id=9ea4cc94eaf84d0c8fec375d056242eb)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:56.943766 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f3b59-5638-4456-ac5a-73d6135c9d24 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:56.946930 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 209/427] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:56 2022] GET /identity/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:56.947851 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f3b59-5638-4456-ac5a-73d6135c9d24 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:56.978312 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:57.018121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 219/428] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:57.019206 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58606f-9305-43cc-a34f-e3525dc78175 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:57.061630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f3b59-5638-4456-ac5a-73d6135c9d24 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:57.100396 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa027732-a323-4db6-b2a4-890fc8ef5b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:57.101300 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa027732-a323-4db6-b2a4-890fc8ef5b87 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:57.101789 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa027732-a323-4db6-b2a4-890fc8ef5b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:57.102493 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa027732-a323-4db6-b2a4-890fc8ef5b87 None admin] PATH_INFO: `/v3/projects/2375c71530404fc591cb968d121ae21e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:57.104150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa027732-a323-4db6-b2a4-890fc8ef5b87 None admin] RBAC: Authorizing `identity:get_project(project_id=2375c71530404fc591cb968d121ae21e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:57.105847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa027732-a323-4db6-b2a4-890fc8ef5b87 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:57.116366 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 210/429] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:56 2022] GET /identity/v3/projects/2375c71530404fc591cb968d121ae21e => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:57.122870 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa027732-a323-4db6-b2a4-890fc8ef5b87 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:57.125754 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58606f-9305-43cc-a34f-e3525dc78175 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:57.154669 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:57.162531 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f540f22-e993-4e19-9838-5fe09522590b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:57.169709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f540f22-e993-4e19-9838-5fe09522590b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:57.169709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f540f22-e993-4e19-9838-5fe09522590b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:57.169709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f540f22-e993-4e19-9838-5fe09522590b None admin] PATH_INFO: `/v3/groups/109db92706954923b3c4932e0a7d9e5e` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:57.170676 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f540f22-e993-4e19-9838-5fe09522590b None admin] RBAC: Authorizing `identity:get_group(group_id=109db92706954923b3c4932e0a7d9e5e)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:57.172246 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f540f22-e993-4e19-9838-5fe09522590b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:57.178271 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 220/430] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Sep 1 16:17:57 2022] GET /identity/v3/groups/109db92706954923b3c4932e0a7d9e5e => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:57.185333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f540f22-e993-4e19-9838-5fe09522590b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:57.295365 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f540f22-e993-4e19-9838-5fe09522590b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:57.331165 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dab24f-c30b-44ef-978e-358971e4b30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:57.332102 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dab24f-c30b-44ef-978e-358971e4b30e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:57.332617 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dab24f-c30b-44ef-978e-358971e4b30e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:57.333094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dab24f-c30b-44ef-978e-358971e4b30e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:57.334020 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dab24f-c30b-44ef-978e-358971e4b30e None admin] RBAC: Adding query filter params (group.id=109db92706954923b3c4932e0a7d9e5e, role.id=42db5f955f2644b1bb22b0a68f8cd829, scope.project.id=9ea4cc94eaf84d0c8fec375d056242eb) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:57.334556 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dab24f-c30b-44ef-978e-358971e4b30e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:57.336324 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dab24f-c30b-44ef-978e-358971e4b30e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:57.343895 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 221/431] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Sep 1 16:17:57 2022] GET /identity/v3/role_assignments?group.id=109db92706954923b3c4932e0a7d9e5e&scope.project.id=9ea4cc94eaf84d0c8fec375d056242eb&role.id=42db5f955f2644b1bb22b0a68f8cd829 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:57.382280 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:57.382714 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dab24f-c30b-44ef-978e-358971e4b30e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:57.393629 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa027732-a323-4db6-b2a4-890fc8ef5b87 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:57.438485 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777c52e-758c-43c8-8058-5aa38b904ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:57.439185 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777c52e-758c-43c8-8058-5aa38b904ef2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:57.439471 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777c52e-758c-43c8-8058-5aa38b904ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:57.439855 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777c52e-758c-43c8-8058-5aa38b904ef2 None admin] PATH_INFO: `/v3/groups/109db92706954923b3c4932e0a7d9e5e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:57.441343 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0777c52e-758c-43c8-8058-5aa38b904ef2 None admin] RBAC: Authorizing `identity:get_group(group_id=109db92706954923b3c4932e0a7d9e5e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:57.442819 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0777c52e-758c-43c8-8058-5aa38b904ef2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:57.450267 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 211/432] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Sep 1 16:17:57 2022] GET /identity/v3/groups/109db92706954923b3c4932e0a7d9e5e => generated 226 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:57.451802 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c13ff-3eb6-459b-842c-5a1cc4892ebc None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:57.452093 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c13ff-3eb6-459b-842c-5a1cc4892ebc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:57.452372 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c13ff-3eb6-459b-842c-5a1cc4892ebc None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:57.454041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 212/433] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:57.455524 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c13ff-3eb6-459b-842c-5a1cc4892ebc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:57.462905 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:57.516207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dab24f-c30b-44ef-978e-358971e4b30e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:57.557418 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e43d344-4ded-43cf-a083-ea28345ee83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:57.558595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43d344-4ded-43cf-a083-ea28345ee83e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:57.559097 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43d344-4ded-43cf-a083-ea28345ee83e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:57.559725 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43d344-4ded-43cf-a083-ea28345ee83e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:57.571353 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e43d344-4ded-43cf-a083-ea28345ee83e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:57.576927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-3e43d344-4ded-43cf-a083-ea28345ee83e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:17:57.576927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:57.576927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:57.576927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:57.576927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:57.576927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:57.576927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:57.576927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:57.576927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:57.576927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:57.576927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:57.576927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:57.576927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:57.576927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:57.576927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:57.576927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:57.577861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:57.577861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:57.577861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:57.577861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:57.577861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:57.577861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:57.577861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:57.577861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:57.577861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:57.577861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:57.577861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:57.577861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:57.577861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:57.577861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:57.577861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:57.577861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:57.578750 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:57.578750 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:57.578750 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:57.578750 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:57.578750 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:57.578750 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:57.578750 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:57.578750 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:57.578750 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:57.578750 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:57.578750 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:17:57.578750 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:17:57.578750 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 222/434] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Sep 1 16:17:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:57.580022 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195033fb-465d-471b-b252-701f97865fc8 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:57.580022 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195033fb-465d-471b-b252-701f97865fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:57.580022 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195033fb-465d-471b-b252-701f97865fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:57.591487 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c13ff-3eb6-459b-842c-5a1cc4892ebc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:57.603735 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-195033fb-465d-471b-b252-701f97865fc8 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:57.605866 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-195033fb-465d-471b-b252-701f97865fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:57.634786 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e7eb1-ddc0-487f-bc64-9e9f16cf5250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:57.635756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e7eb1-ddc0-487f-bc64-9e9f16cf5250 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:57.636146 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e7eb1-ddc0-487f-bc64-9e9f16cf5250 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:57.636426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e7eb1-ddc0-487f-bc64-9e9f16cf5250 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:57.637274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e7eb1-ddc0-487f-bc64-9e9f16cf5250 None admin] RBAC: Adding query filter params (group.id=109db92706954923b3c4932e0a7d9e5e, role.id=42db5f955f2644b1bb22b0a68f8cd829, scope.project.id=2375c71530404fc591cb968d121ae21e) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:57.637585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e7eb1-ddc0-487f-bc64-9e9f16cf5250 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:57.639264 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e7eb1-ddc0-487f-bc64-9e9f16cf5250 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:57.646032 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 213/435] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Sep 1 16:17:57 2022] GET /identity/v3/role_assignments?group.id=109db92706954923b3c4932e0a7d9e5e&scope.project.id=2375c71530404fc591cb968d121ae21e&role.id=42db5f955f2644b1bb22b0a68f8cd829 => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:57.646879 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e7eb1-ddc0-487f-bc64-9e9f16cf5250 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:57.679798 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:57.693597 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 223/436] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:57.794835 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e7eb1-ddc0-487f-bc64-9e9f16cf5250 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:57.834604 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443e024-0637-4599-a4a4-12d39cb1db43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:57.835036 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443e024-0637-4599-a4a4-12d39cb1db43 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:57.835322 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443e024-0637-4599-a4a4-12d39cb1db43 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:57.835599 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443e024-0637-4599-a4a4-12d39cb1db43 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:57.836282 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443e024-0637-4599-a4a4-12d39cb1db43 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:57.836587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443e024-0637-4599-a4a4-12d39cb1db43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:57.838944 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443e024-0637-4599-a4a4-12d39cb1db43 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:57.849273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 214/437] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:17:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:57.855512 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195033fb-465d-471b-b252-701f97865fc8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:57.921554 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:57.978278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195033fb-465d-471b-b252-701f97865fc8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:57.988931 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443e024-0637-4599-a4a4-12d39cb1db43 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:58.026274 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0977e3-198e-488b-84c1-a43e6487df5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:58.027209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0977e3-198e-488b-84c1-a43e6487df5d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:58.027726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0977e3-198e-488b-84c1-a43e6487df5d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:58.027912 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0977e3-198e-488b-84c1-a43e6487df5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:58.099383 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:58.111787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0977e3-198e-488b-84c1-a43e6487df5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:58.113335 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0977e3-198e-488b-84c1-a43e6487df5d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:58.114178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443e024-0637-4599-a4a4-12d39cb1db43 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:58.148967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488d009-268a-4a81-8096-7ef0609e82b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:58.149959 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488d009-268a-4a81-8096-7ef0609e82b0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:58.150284 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488d009-268a-4a81-8096-7ef0609e82b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:58.150616 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488d009-268a-4a81-8096-7ef0609e82b0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:58.161033 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488d009-268a-4a81-8096-7ef0609e82b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:58.163643 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:58.166873 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-3488d009-268a-4a81-8096-7ef0609e82b0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:17:58.166873 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:58.166873 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:58.166873 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:58.166873 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:58.166873 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:58.166873 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:58.166873 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:58.166873 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:58.166873 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:58.166873 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:58.166873 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:58.166873 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:58.166873 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:58.166873 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:58.166873 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:58.168071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:58.168071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:58.168071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:58.168071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:58.168071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:58.168071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:58.168071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:58.168071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:58.168071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:58.168071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:58.168071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:58.168071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:58.168071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:58.168071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:58.168071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:58.168071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:58.169942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:58.169942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:58.169942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:58.169942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:58.169942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:58.169942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:58.169942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:58.169942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:58.169942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:58.169942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:58.169942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:17:58.169942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:17:58.169942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 215/438] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Sep 1 16:17:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:58.174276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488d009-268a-4a81-8096-7ef0609e82b0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:58.222372 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 224/439] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:17:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:17:58.227864 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0977e3-198e-488b-84c1-a43e6487df5d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:58.283223 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488d009-268a-4a81-8096-7ef0609e82b0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:58.320732 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476822a5-7cdb-40a3-b29d-3c8b8d508f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:58.321858 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476822a5-7cdb-40a3-b29d-3c8b8d508f63 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:58.322135 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476822a5-7cdb-40a3-b29d-3c8b8d508f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:58.322428 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476822a5-7cdb-40a3-b29d-3c8b8d508f63 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:58.323092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476822a5-7cdb-40a3-b29d-3c8b8d508f63 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:58.323393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476822a5-7cdb-40a3-b29d-3c8b8d508f63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:58.324772 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476822a5-7cdb-40a3-b29d-3c8b8d508f63 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:58.332671 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0977e3-198e-488b-84c1-a43e6487df5d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:58.333006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 216/440] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:17:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:58.334257 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2af6c-5d74-43c8-9bce-74612bd96580 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:58.334549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2af6c-5d74-43c8-9bce-74612bd96580 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:58.334826 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2af6c-5d74-43c8-9bce-74612bd96580 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:58.336013 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 217/441] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:17:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:58.338783 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf2af6c-5d74-43c8-9bce-74612bd96580 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:58.350844 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:58.371151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb6ff5c-2ccb-4ab0-8e01-2f4035b584ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:58.371774 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6ff5c-2ccb-4ab0-8e01-2f4035b584ca None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:58.372052 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6ff5c-2ccb-4ab0-8e01-2f4035b584ca None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:58.372322 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6ff5c-2ccb-4ab0-8e01-2f4035b584ca None admin] PATH_INFO: `/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:58.373422 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb6ff5c-2ccb-4ab0-8e01-2f4035b584ca None admin] RBAC: Authorizing `identity:get_project(project_id=9ea4cc94eaf84d0c8fec375d056242eb)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:58.374896 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb6ff5c-2ccb-4ab0-8e01-2f4035b584ca None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:58.376501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 225/442] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:58 2022] GET /identity/v3/projects/9ea4cc94eaf84d0c8fec375d056242eb => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:58.377586 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44762127-ae13-4949-aec5-eaac563cc1f4 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:58.377880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44762127-ae13-4949-aec5-eaac563cc1f4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:58.378130 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44762127-ae13-4949-aec5-eaac563cc1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:58.395901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-44762127-ae13-4949-aec5-eaac563cc1f4 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:58.397095 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-44762127-ae13-4949-aec5-eaac563cc1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:58.407583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:58.442802 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf2af6c-5d74-43c8-9bce-74612bd96580 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:58.464993 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 226/443] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:17:58.465813 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44762127-ae13-4949-aec5-eaac563cc1f4 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:58.481042 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52237e9c-b127-4561-a955-99737db5ea15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:58.481639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52237e9c-b127-4561-a955-99737db5ea15 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:58.481935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52237e9c-b127-4561-a955-99737db5ea15 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:58.482210 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52237e9c-b127-4561-a955-99737db5ea15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:58.554971 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52237e9c-b127-4561-a955-99737db5ea15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:58.556409 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52237e9c-b127-4561-a955-99737db5ea15 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:58.568558 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:58.576860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44762127-ae13-4949-aec5-eaac563cc1f4 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:58.612174 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7723b0-9a9d-4d4b-b98d-73086d679e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:58.612792 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7723b0-9a9d-4d4b-b98d-73086d679e6a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:58.613072 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7723b0-9a9d-4d4b-b98d-73086d679e6a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:58.613345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7723b0-9a9d-4d4b-b98d-73086d679e6a None admin] PATH_INFO: `/v3/groups/109db92706954923b3c4932e0a7d9e5e` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:58.614483 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7723b0-9a9d-4d4b-b98d-73086d679e6a None admin] RBAC: Authorizing `identity:get_group(group_id=109db92706954923b3c4932e0a7d9e5e)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:58.615946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7723b0-9a9d-4d4b-b98d-73086d679e6a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:58.617661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 227/444] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Sep 1 16:17:58 2022] GET /identity/v3/groups/109db92706954923b3c4932e0a7d9e5e => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:58.623133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7723b0-9a9d-4d4b-b98d-73086d679e6a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:58.639155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:58.664921 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 218/445] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:17:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 16:17:58.671251 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52237e9c-b127-4561-a955-99737db5ea15 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:58.734512 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7723b0-9a9d-4d4b-b98d-73086d679e6a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:58.771421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eece06-fc51-4375-88b9-03df9cf15d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:58.771933 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52237e9c-b127-4561-a955-99737db5ea15 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:58.772179 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eece06-fc51-4375-88b9-03df9cf15d0c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:58.772524 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eece06-fc51-4375-88b9-03df9cf15d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:58.772863 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eece06-fc51-4375-88b9-03df9cf15d0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:58.773655 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eece06-fc51-4375-88b9-03df9cf15d0c None admin] RBAC: Adding query filter params (group.id=109db92706954923b3c4932e0a7d9e5e, role.id=72700efc336a4b4f992b08ff9ea99f0c, scope.project.id=9ea4cc94eaf84d0c8fec375d056242eb) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:58.774033 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eece06-fc51-4375-88b9-03df9cf15d0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:58.775479 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eece06-fc51-4375-88b9-03df9cf15d0c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:58.780993 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 228/446] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Sep 1 16:17:58 2022] GET /identity/v3/role_assignments?group.id=109db92706954923b3c4932e0a7d9e5e&scope.project.id=9ea4cc94eaf84d0c8fec375d056242eb&role.id=72700efc336a4b4f992b08ff9ea99f0c => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:58.781757 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eece06-fc51-4375-88b9-03df9cf15d0c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:58.783025 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:58.814502 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5317ea-c162-4579-80ae-bc7e3ada62fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:58.815242 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5317ea-c162-4579-80ae-bc7e3ada62fc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:58.815628 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5317ea-c162-4579-80ae-bc7e3ada62fc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:58.815912 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5317ea-c162-4579-80ae-bc7e3ada62fc None admin] PATH_INFO: `/v3/projects/2375c71530404fc591cb968d121ae21e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:58.817526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5317ea-c162-4579-80ae-bc7e3ada62fc None admin] RBAC: Authorizing `identity:get_project(project_id=2375c71530404fc591cb968d121ae21e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:58.819016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5317ea-c162-4579-80ae-bc7e3ada62fc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:58.820847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 219/447] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:58 2022] GET /identity/v3/projects/2375c71530404fc591cb968d121ae21e => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:58.826451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5317ea-c162-4579-80ae-bc7e3ada62fc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:58.871792 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:58.901033 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eece06-fc51-4375-88b9-03df9cf15d0c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:58.941165 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea201c-9091-40bc-9444-831338b54592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:58.941779 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea201c-9091-40bc-9444-831338b54592 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:58.942067 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea201c-9091-40bc-9444-831338b54592 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:58.942379 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea201c-9091-40bc-9444-831338b54592 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:58.946640 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5317ea-c162-4579-80ae-bc7e3ada62fc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:58.958847 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea201c-9091-40bc-9444-831338b54592 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:17:58.963928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-9aea201c-9091-40bc-9444-831338b54592 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:17:58.963928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:58.963928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:58.963928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:58.963928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:58.963928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:58.963928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:58.963928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:58.963928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:58.963928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:58.963928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:58.963928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:58.963928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:17:58.963928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:17:58.963928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:17:58.963928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:17:58.965018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:17:58.965018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:17:58.965018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:17:58.965018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:17:58.965018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:58.965018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:58.965018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:58.965018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:58.965018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:58.965018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:58.965018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:58.965018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:58.965018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:58.965018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:58.965018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:58.965018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:58.965921 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:58.965921 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:58.965921 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:58.965921 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:58.965921 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:17:58.965921 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:17:58.965921 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:17:58.965921 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:17:58.965921 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:17:58.965921 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:17:58.965921 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:17:58.965921 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:17:58.965921 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 229/448] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:17:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:58.971197 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea201c-9091-40bc-9444-831338b54592 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:58.989525 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a8e33-bfd8-47b3-8ae1-3cd1cc504ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:58.990162 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a8e33-bfd8-47b3-8ae1-3cd1cc504ebe None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:58.990473 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a8e33-bfd8-47b3-8ae1-3cd1cc504ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:58.990780 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a8e33-bfd8-47b3-8ae1-3cd1cc504ebe None admin] PATH_INFO: `/v3/groups/109db92706954923b3c4932e0a7d9e5e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:58.991888 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a8e33-bfd8-47b3-8ae1-3cd1cc504ebe None admin] RBAC: Authorizing `identity:get_group(group_id=109db92706954923b3c4932e0a7d9e5e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:58.993887 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a8e33-bfd8-47b3-8ae1-3cd1cc504ebe None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:59.001510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 220/449] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Sep 1 16:17:58 2022] GET /identity/v3/groups/109db92706954923b3c4932e0a7d9e5e => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:59.001510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a8e33-bfd8-47b3-8ae1-3cd1cc504ebe None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:59.067223 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:59.091739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea201c-9091-40bc-9444-831338b54592 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:59.132122 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a8e33-bfd8-47b3-8ae1-3cd1cc504ebe None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:59.134481 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29baeeb7-93f3-4ebd-a57a-ae1a3c9443cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:59.137049 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29baeeb7-93f3-4ebd-a57a-ae1a3c9443cd None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:59.137049 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29baeeb7-93f3-4ebd-a57a-ae1a3c9443cd None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:59.137049 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29baeeb7-93f3-4ebd-a57a-ae1a3c9443cd None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:59.137049 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29baeeb7-93f3-4ebd-a57a-ae1a3c9443cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:59.137049 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29baeeb7-93f3-4ebd-a57a-ae1a3c9443cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:59.137953 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29baeeb7-93f3-4ebd-a57a-ae1a3c9443cd None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:59.143578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:59.148329 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 230/450] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:17:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:59.153042 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29baeeb7-93f3-4ebd-a57a-ae1a3c9443cd None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:59.177343 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b65a0c-ad36-436d-bca9-b877e7cbe951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:59.177948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b65a0c-ad36-436d-bca9-b877e7cbe951 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:59.178271 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b65a0c-ad36-436d-bca9-b877e7cbe951 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:59.178548 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b65a0c-ad36-436d-bca9-b877e7cbe951 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:59.179273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b65a0c-ad36-436d-bca9-b877e7cbe951 None admin] RBAC: Adding query filter params (group.id=109db92706954923b3c4932e0a7d9e5e, role.id=72700efc336a4b4f992b08ff9ea99f0c, scope.project.id=2375c71530404fc591cb968d121ae21e) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:59.179580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b65a0c-ad36-436d-bca9-b877e7cbe951 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:59.181551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b65a0c-ad36-436d-bca9-b877e7cbe951 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:59.188265 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 221/451] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Sep 1 16:17:59 2022] GET /identity/v3/role_assignments?group.id=109db92706954923b3c4932e0a7d9e5e&scope.project.id=2375c71530404fc591cb968d121ae21e&role.id=72700efc336a4b4f992b08ff9ea99f0c => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:59.270601 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29baeeb7-93f3-4ebd-a57a-ae1a3c9443cd None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:59.310631 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79cfb1-8eb4-4d17-b062-0fac7c7aea08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:59.311503 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79cfb1-8eb4-4d17-b062-0fac7c7aea08 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:59.311938 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79cfb1-8eb4-4d17-b062-0fac7c7aea08 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:59.312209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79cfb1-8eb4-4d17-b062-0fac7c7aea08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:59.381380 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:59.384775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79cfb1-8eb4-4d17-b062-0fac7c7aea08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:59.386778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79cfb1-8eb4-4d17-b062-0fac7c7aea08 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:59.486821 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 231/452] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:17:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 16:17:59.492345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b65a0c-ad36-436d-bca9-b877e7cbe951 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:59.596757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b65a0c-ad36-436d-bca9-b877e7cbe951 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:59.606581 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:59.629157 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeafe88-b9cc-4c76-97a2-e8d36b9fa3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:59.629773 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeafe88-b9cc-4c76-97a2-e8d36b9fa3ec None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:59.630059 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeafe88-b9cc-4c76-97a2-e8d36b9fa3ec None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:59.630369 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeafe88-b9cc-4c76-97a2-e8d36b9fa3ec None admin] PATH_INFO: `/v3/projects/e2c38221adde4053b2b37201b4051903` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:59.631523 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeafe88-b9cc-4c76-97a2-e8d36b9fa3ec None admin] RBAC: Authorizing `identity:get_project(project_id=e2c38221adde4053b2b37201b4051903)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:59.632921 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeafe88-b9cc-4c76-97a2-e8d36b9fa3ec None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:59.635058 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 222/453] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:17:59 2022] GET /identity/v3/projects/e2c38221adde4053b2b37201b4051903 => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:17:59.640633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79cfb1-8eb4-4d17-b062-0fac7c7aea08 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:59.732681 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:59.735567 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79cfb1-8eb4-4d17-b062-0fac7c7aea08 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:59.768844 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da483545-9ace-4dcf-ae93-2a47fd299432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:59.769451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da483545-9ace-4dcf-ae93-2a47fd299432 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:59.769742 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da483545-9ace-4dcf-ae93-2a47fd299432 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:59.769996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da483545-9ace-4dcf-ae93-2a47fd299432 None admin] PATH_INFO: `/v3/groups/e473cd65b6624884a56c7e28e2d8ebb9` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:59.771114 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da483545-9ace-4dcf-ae93-2a47fd299432 None admin] RBAC: Authorizing `identity:get_group(group_id=e473cd65b6624884a56c7e28e2d8ebb9)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:59.772533 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da483545-9ace-4dcf-ae93-2a47fd299432 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:59.774059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 232/454] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Sep 1 16:17:59 2022] GET /identity/v3/groups/e473cd65b6624884a56c7e28e2d8ebb9 => generated 229 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:59.780418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeafe88-b9cc-4c76-97a2-e8d36b9fa3ec None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:59.875814 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeafe88-b9cc-4c76-97a2-e8d36b9fa3ec None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:59.907289 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f0735-5a04-4f06-ba43-3e8285b28861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:59.907827 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f0735-5a04-4f06-ba43-3e8285b28861 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:59.908103 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f0735-5a04-4f06-ba43-3e8285b28861 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:59.908375 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f0735-5a04-4f06-ba43-3e8285b28861 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:59.909081 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f0735-5a04-4f06-ba43-3e8285b28861 None admin] RBAC: Adding query filter params (group.id=e473cd65b6624884a56c7e28e2d8ebb9, role.id=ff293acb8551463aafe6970230042b89, scope.project.id=e2c38221adde4053b2b37201b4051903) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:59.909387 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f0735-5a04-4f06-ba43-3e8285b28861 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:59.910834 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f0735-5a04-4f06-ba43-3e8285b28861 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:59.916465 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 223/455] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Sep 1 16:17:59 2022] GET /identity/v3/role_assignments?group.id=e473cd65b6624884a56c7e28e2d8ebb9&scope.project.id=e2c38221adde4053b2b37201b4051903&role.id=ff293acb8551463aafe6970230042b89 => generated 611 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:02.368917 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65330122-e2b3-45dc-ba7d-455e63e81bb4 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:02.369402 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65330122-e2b3-45dc-ba7d-455e63e81bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:02.369402 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65330122-e2b3-45dc-ba7d-455e63e81bb4 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:02.370988 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 233/456] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:02.376375 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290bf5af-ff8c-44dd-b35b-a9e7ffc898c7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:02.376782 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290bf5af-ff8c-44dd-b35b-a9e7ffc898c7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:02.376979 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290bf5af-ff8c-44dd-b35b-a9e7ffc898c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:02.395869 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2dacf8-09b0-4dc2-bec3-f9d73e02732b None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:02.396161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2dacf8-09b0-4dc2-bec3-f9d73e02732b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:02.396411 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2dacf8-09b0-4dc2-bec3-f9d73e02732b None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:02.398022 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 234/457] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:18:02.402853 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99bab8-114a-4601-90c8-1164ae7d6763 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:02.403216 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99bab8-114a-4601-90c8-1164ae7d6763 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:02.403407 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99bab8-114a-4601-90c8-1164ae7d6763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:02.406777 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-290bf5af-ff8c-44dd-b35b-a9e7ffc898c7 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:02.407974 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-290bf5af-ff8c-44dd-b35b-a9e7ffc898c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:02.439064 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7a99bab8-114a-4601-90c8-1164ae7d6763 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:02.440251 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7a99bab8-114a-4601-90c8-1164ae7d6763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:02.465190 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:02.512158 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 224/458] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:18:02.535837 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cecb76-103e-437f-9c92-b0dcf467aec6 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:02.536133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cecb76-103e-437f-9c92-b0dcf467aec6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:02.536534 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cecb76-103e-437f-9c92-b0dcf467aec6 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:02.540979 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 225/459] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:18:02.550422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac752643-2a8e-497a-bed7-51ee828658f2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:02.550785 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac752643-2a8e-497a-bed7-51ee828658f2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:02.551134 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac752643-2a8e-497a-bed7-51ee828658f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:02.552486 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 235/460] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:18:02.587221 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ac752643-2a8e-497a-bed7-51ee828658f2 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:02.588413 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ac752643-2a8e-497a-bed7-51ee828658f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:02.685220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 226/461] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:18:02.844765 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99bab8-114a-4601-90c8-1164ae7d6763 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:02.867605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac752643-2a8e-497a-bed7-51ee828658f2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:02.955109 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99bab8-114a-4601-90c8-1164ae7d6763 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:02.958497 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:02.978236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac752643-2a8e-497a-bed7-51ee828658f2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:02.978236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:03.016610 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f1957-9884-4848-99a6-a98405816b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:03.017253 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f1957-9884-4848-99a6-a98405816b2a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:03.017531 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f1957-9884-4848-99a6-a98405816b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:03.017806 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f1957-9884-4848-99a6-a98405816b2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:03.018928 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77240a7-b6e9-411f-b636-e3d4f39ab469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:03.019549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77240a7-b6e9-411f-b636-e3d4f39ab469 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:03.019926 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77240a7-b6e9-411f-b636-e3d4f39ab469 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:03.020197 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77240a7-b6e9-411f-b636-e3d4f39ab469 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:03.030172 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f1957-9884-4848-99a6-a98405816b2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:03.031150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77240a7-b6e9-411f-b636-e3d4f39ab469 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:03.031658 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f1957-9884-4848-99a6-a98405816b2a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:03.032594 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77240a7-b6e9-411f-b636-e3d4f39ab469 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:03.048817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-a77240a7-b6e9-411f-b636-e3d4f39ab469 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:03.048817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:03.048817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:03.048817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:03.048817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:03.048817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:03.048817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:03.048817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:03.048817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:03.048817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:03.048817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:03.048817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:03.048817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:03.048817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:03.048817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:03.048817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:03.049962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:03.049962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:03.049962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:03.049962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:03.049962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:03.049962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:03.049962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:03.049962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:03.049962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:03.049962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:03.049962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:03.049962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:03.049962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:03.049962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:03.049962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:03.049962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:03.049962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:03.051069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:03.051069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:03.051069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:03.051069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 227/462] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:18:03.051350 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0937f35-8406-4708-8f96-837d711fe4a1 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:03.051430 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0937f35-8406-4708-8f96-837d711fe4a1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:03.051819 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0937f35-8406-4708-8f96-837d711fe4a1 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:03.052877 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 228/463] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:03.058935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0937f35-8406-4708-8f96-837d711fe4a1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:03.066546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-fc2f1957-9884-4848-99a6-a98405816b2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:03.066546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:03.066546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:03.066546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:03.066546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:03.066546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:03.066546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:03.066546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:03.066546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:03.066546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:03.066546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:03.066546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:03.066546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:03.066546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:03.066546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:03.066546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:03.067654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:03.067654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:03.067654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:03.067654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:03.067654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:03.067654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:03.067654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:03.067654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:03.067654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:03.067654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:03.067654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:03.067654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:03.067654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:03.067654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:03.067654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:03.067654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:03.067654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:03.068599 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:03.068599 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:03.068599 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:03.070691 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 236/464] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:03.072700 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1569301-d5b4-40b6-958c-2f98b7c2f280 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:03.073212 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1569301-d5b4-40b6-958c-2f98b7c2f280 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:03.073657 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1569301-d5b4-40b6-958c-2f98b7c2f280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:03.092761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e1569301-d5b4-40b6-958c-2f98b7c2f280 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:03.093972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e1569301-d5b4-40b6-958c-2f98b7c2f280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:03.163849 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 237/465] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:18:03.170013 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1569301-d5b4-40b6-958c-2f98b7c2f280 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:03.181657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0937f35-8406-4708-8f96-837d711fe4a1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:03.219790 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f19e62a-e8d4-42a0-9971-94e13e22428a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:03.220316 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19e62a-e8d4-42a0-9971-94e13e22428a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:03.220599 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19e62a-e8d4-42a0-9971-94e13e22428a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:03.220956 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19e62a-e8d4-42a0-9971-94e13e22428a None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:03.221606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f19e62a-e8d4-42a0-9971-94e13e22428a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:03.221913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f19e62a-e8d4-42a0-9971-94e13e22428a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:03.223315 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f19e62a-e8d4-42a0-9971-94e13e22428a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:03.240654 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 229/466] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:18:03.240654 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f19e62a-e8d4-42a0-9971-94e13e22428a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:03.257932 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:03.284455 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:03.311613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1569301-d5b4-40b6-958c-2f98b7c2f280 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:03.356839 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1386f-6a9a-43fc-857f-26d0a5d793ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:03.358894 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1386f-6a9a-43fc-857f-26d0a5d793ca None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:03.359185 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1386f-6a9a-43fc-857f-26d0a5d793ca None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:03.359393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f19e62a-e8d4-42a0-9971-94e13e22428a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:03.359497 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1386f-6a9a-43fc-857f-26d0a5d793ca None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:03.360112 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb1386f-6a9a-43fc-857f-26d0a5d793ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:03.360421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb1386f-6a9a-43fc-857f-26d0a5d793ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:03.362135 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb1386f-6a9a-43fc-857f-26d0a5d793ca None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:03.381636 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 238/467] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:03.381636 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1386f-6a9a-43fc-857f-26d0a5d793ca None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:03.404469 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4abb60-9f20-4e3d-bbbf-6ba9d6b286b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:03.405740 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4abb60-9f20-4e3d-bbbf-6ba9d6b286b8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:03.406249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4abb60-9f20-4e3d-bbbf-6ba9d6b286b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:03.406725 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4abb60-9f20-4e3d-bbbf-6ba9d6b286b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:03.418907 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4abb60-9f20-4e3d-bbbf-6ba9d6b286b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:03.420773 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4abb60-9f20-4e3d-bbbf-6ba9d6b286b8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:03.437545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-eb4abb60-9f20-4e3d-bbbf-6ba9d6b286b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:03.437545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:03.437545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:03.437545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:03.437545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:03.437545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:03.437545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:03.437545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:03.437545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:03.437545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:03.437545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:03.437545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:03.437545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:03.437545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:03.437545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:03.437545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:03.438510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:03.438510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:03.438510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:03.438510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:03.438510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:03.438510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:03.438510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:03.438510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:03.438510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:03.438510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:03.438510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:03.438510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:03.438510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:03.438510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:03.438510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:03.438510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:03.438510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:03.439396 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:03.439396 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:03.439396 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:03.441493 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 230/468] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:03.442492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4abb60-9f20-4e3d-bbbf-6ba9d6b286b8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:03.496432 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1386f-6a9a-43fc-857f-26d0a5d793ca None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:03.513960 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:03.532415 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472706d-d94d-4664-b554-4b28c60071b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:03.533017 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472706d-d94d-4664-b554-4b28c60071b6 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:03.533314 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472706d-d94d-4664-b554-4b28c60071b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:03.533586 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472706d-d94d-4664-b554-4b28c60071b6 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:03.534255 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b472706d-d94d-4664-b554-4b28c60071b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:03.535704 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b472706d-d94d-4664-b554-4b28c60071b6 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:03.547960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:03.557640 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4abb60-9f20-4e3d-bbbf-6ba9d6b286b8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:03.563444 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 239/469] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Sep 1 16:18:03 2022] POST /identity/v3/users => generated 246 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:18:03.564495 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472706d-d94d-4664-b554-4b28c60071b6 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:03.616467 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db4819-b39e-44e2-ba27-4235a543fc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:03.617419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db4819-b39e-44e2-ba27-4235a543fc3e None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:03.618655 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db4819-b39e-44e2-ba27-4235a543fc3e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:03.618978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db4819-b39e-44e2-ba27-4235a543fc3e None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:03.620341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13db4819-b39e-44e2-ba27-4235a543fc3e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:03.621858 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13db4819-b39e-44e2-ba27-4235a543fc3e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:03.651268 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 231/470] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Sep 1 16:18:03 2022] POST /identity/v3/users => generated 248 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:18:03.658136 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db4819-b39e-44e2-ba27-4235a543fc3e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:03.717868 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472706d-d94d-4664-b554-4b28c60071b6 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:03.762520 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59f11b-01f5-4592-9b76-55ebead2adb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:03.763071 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59f11b-01f5-4592-9b76-55ebead2adb0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:03.763507 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59f11b-01f5-4592-9b76-55ebead2adb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:03.763788 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59f11b-01f5-4592-9b76-55ebead2adb0 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:03.764651 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59f11b-01f5-4592-9b76-55ebead2adb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:03.764962 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59f11b-01f5-4592-9b76-55ebead2adb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:03.767329 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59f11b-01f5-4592-9b76-55ebead2adb0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:03.782616 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 240/471] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:03.789208 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59f11b-01f5-4592-9b76-55ebead2adb0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:03.832360 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db4819-b39e-44e2-ba27-4235a543fc3e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:03.865424 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:03.869076 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:03.883968 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb7101-6106-4191-9721-208f109e8610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:03.884627 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb7101-6106-4191-9721-208f109e8610 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:03.884911 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb7101-6106-4191-9721-208f109e8610 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:03.885183 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb7101-6106-4191-9721-208f109e8610 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:03.900168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb7101-6106-4191-9721-208f109e8610 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:03.902468 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb7101-6106-4191-9721-208f109e8610 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:03.923089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-5edb7101-6106-4191-9721-208f109e8610 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:03.923089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:03.923089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:03.923089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:03.923089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:03.923089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:03.923089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:03.923089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:03.923089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:03.923089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:03.923089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:03.923089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:03.923089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:03.923089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:03.923089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:03.923089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:03.924049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:03.924049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:03.924049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:03.924049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:03.924049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:03.924049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:03.924049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:03.924049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:03.924049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:03.924049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:03.924049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:03.924049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:03.924049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:03.924049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:03.924049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:03.924049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:03.924049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:03.925758 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:03.925758 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:03.925758 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:03.925962 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59f11b-01f5-4592-9b76-55ebead2adb0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:03.926047 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 232/472] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:03.932697 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb7101-6106-4191-9721-208f109e8610 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:03.974311 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b5c71a-59ba-405d-98a4-39b57c296ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:03.975764 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5c71a-59ba-405d-98a4-39b57c296ee6 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:03.976326 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5c71a-59ba-405d-98a4-39b57c296ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:03.976769 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5c71a-59ba-405d-98a4-39b57c296ee6 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:03.980871 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b5c71a-59ba-405d-98a4-39b57c296ee6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:03.981430 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b5c71a-59ba-405d-98a4-39b57c296ee6 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:04.015915 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 241/473] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Sep 1 16:18:03 2022] POST /identity/v3/users => generated 248 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:18:04.036818 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2cf23-e781-4c45-a702-0043561a671b None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:04.037505 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2cf23-e781-4c45-a702-0043561a671b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:04.038031 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2cf23-e781-4c45-a702-0043561a671b None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:04.040412 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 242/474] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:18:04.050389 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd8928-2d9d-45d2-b066-03a4407b420c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:04.050986 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd8928-2d9d-45d2-b066-03a4407b420c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:04.051330 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd8928-2d9d-45d2-b066-03a4407b420c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:04.074044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-72fd8928-2d9d-45d2-b066-03a4407b420c None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:04.075480 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-72fd8928-2d9d-45d2-b066-03a4407b420c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:04.083636 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb7101-6106-4191-9721-208f109e8610 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:04.124790 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381f6b4-f811-44a2-a406-9db5cdc373fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:04.125733 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381f6b4-f811-44a2-a406-9db5cdc373fd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:04.126218 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381f6b4-f811-44a2-a406-9db5cdc373fd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:04.126821 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381f6b4-f811-44a2-a406-9db5cdc373fd None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:04.128201 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3381f6b4-f811-44a2-a406-9db5cdc373fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:04.128958 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3381f6b4-f811-44a2-a406-9db5cdc373fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:04.131263 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3381f6b4-f811-44a2-a406-9db5cdc373fd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:04.148909 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 233/475] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:04.159329 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381f6b4-f811-44a2-a406-9db5cdc373fd None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:04.161155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 243/476] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:18:04.202106 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:04.279953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381f6b4-f811-44a2-a406-9db5cdc373fd None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:04.321677 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f834049-9e4b-40ca-8295-8918e5a0f2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:04.322319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f834049-9e4b-40ca-8295-8918e5a0f2ce None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:04.322604 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f834049-9e4b-40ca-8295-8918e5a0f2ce None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:04.322878 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f834049-9e4b-40ca-8295-8918e5a0f2ce None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:04.323533 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f834049-9e4b-40ca-8295-8918e5a0f2ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:04.325031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f834049-9e4b-40ca-8295-8918e5a0f2ce None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:04.358771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 234/477] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Sep 1 16:18:04 2022] POST /identity/v3/users => generated 249 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:18:04.505806 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd8928-2d9d-45d2-b066-03a4407b420c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:04.552451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:04.615584 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd8928-2d9d-45d2-b066-03a4407b420c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:04.653019 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4c99e-e3ce-4292-822d-7aa8e1807e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:04.653971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4c99e-e3ce-4292-822d-7aa8e1807e07 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:04.654484 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4c99e-e3ce-4292-822d-7aa8e1807e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:04.654956 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4c99e-e3ce-4292-822d-7aa8e1807e07 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:04.655848 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4c99e-e3ce-4292-822d-7aa8e1807e07 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:04.657428 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4c99e-e3ce-4292-822d-7aa8e1807e07 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:04.666355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.sql.core [None req-2ac4c99e-e3ce-4292-822d-7aa8e1807e07 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 16:18:04.666355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 16:18:04.666355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [parameters: {'id': '49b0f52aa3fa4401ad6208aaa9ebf110', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 16:18:04.666355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81504) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 16:18:04.683383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-2ac4c99e-e3ce-4292-822d-7aa8e1807e07 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 16:18:04.683383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:04.683383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 16:18:04.683383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 16:18:04.683383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 16:18:04.683383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 16:18:04.683383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 16:18:04.683383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 16:18:04.683383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 16:18:04.683383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application conn.query(q) Sep 01 16:18:04.683383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 16:18:04.683383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 16:18:04.683383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 16:18:04.683383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application result.read() Sep 01 16:18:04.683383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 16:18:04.684297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 16:18:04.684297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 16:18:04.684297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 16:18:04.684297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 16:18:04.684297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 16:18:04.684297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 16:18:04.684297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 16:18:04.684297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 16:18:04.684297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:04.684297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 16:18:04.684297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:04.684297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:04.684297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 16:18:04.684297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 16:18:04.684297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 01 16:18:04.684297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return ref.to_dict() Sep 01 16:18:04.684297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 16:18:04.684297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application next(self.gen) Sep 01 16:18:04.685264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 16:18:04.685264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application yield resource Sep 01 16:18:04.685264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 16:18:04.685264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application next(self.gen) Sep 01 16:18:04.685264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 16:18:04.685264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 16:18:04.685264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 16:18:04.685264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 16:18:04.685264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 16:18:04.685264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise self.value Sep 01 16:18:04.685264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 16:18:04.685264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 16:18:04.685264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 16:18:04.685264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application session.commit() Sep 01 16:18:04.685264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 16:18:04.685264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 16:18:04.685264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 16:18:04.686144 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 16:18:04.686144 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 16:18:04.686144 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.session.flush() Sep 01 16:18:04.686144 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 16:18:04.686144 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 16:18:04.686144 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 16:18:04.686144 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 16:18:04.686144 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 16:18:04.686144 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application compat.raise_( Sep 01 16:18:04.686144 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 16:18:04.686144 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception Sep 01 16:18:04.686144 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 16:18:04.686144 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 16:18:04.686144 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 16:18:04.686144 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 16:18:04.686144 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 16:18:04.686144 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 16:18:04.688019 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 16:18:04.688019 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 16:18:04.688019 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 16:18:04.688019 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 16:18:04.688019 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 16:18:04.688019 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 16:18:04.688019 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 16:18:04.688019 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 16:18:04.688019 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 16:18:04.688019 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 16:18:04.688019 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 16:18:04.688019 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 16:18:04.688019 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 16:18:04.688019 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 16:18:04.688019 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 16:18:04.688019 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception Sep 01 16:18:04.688879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 16:18:04.688879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 16:18:04.688879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 16:18:04.688879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 16:18:04.688879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 16:18:04.688879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 16:18:04.688879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 16:18:04.688879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application conn.query(q) Sep 01 16:18:04.688879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 16:18:04.688879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 16:18:04.688879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 16:18:04.688879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application result.read() Sep 01 16:18:04.688879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 16:18:04.688879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 16:18:04.688879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 16:18:04.688879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 16:18:04.689710 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 16:18:04.689710 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 16:18:04.689710 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 16:18:04.689710 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 16:18:04.689710 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 16:18:04.689710 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 16:18:04.689710 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [parameters: {'id': '49b0f52aa3fa4401ad6208aaa9ebf110', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 16:18:04.689710 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 16:18:04.689710 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:04.689710 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:18:04.689710 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:04.689710 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:04.689710 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:04.689710 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:04.689710 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:04.689710 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:04.690583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:04.690583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:04.690583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:04.690583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:04.690583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:04.690583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:04.690583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 01 16:18:04.690583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 01 16:18:04.690583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:04.690583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:04.690583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 01 16:18:04.690583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 01 16:18:04.690583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 16:18:04.690583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 16:18:04.690583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 16:18:04.690583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:04.693880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 244/478] 10.222.0.58 () {60 vars in 1131 bytes} [Thu Sep 1 16:18:04 2022] POST /identity/v3/roles => generated 144 bytes in 188 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:04.700317 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f834049-9e4b-40ca-8295-8918e5a0f2ce None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:04.809504 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:04.822062 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f834049-9e4b-40ca-8295-8918e5a0f2ce None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:04.859449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43a06bf-bf98-48ca-ba72-8a54a017d438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:04.860079 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a06bf-bf98-48ca-ba72-8a54a017d438 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:04.860359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a06bf-bf98-48ca-ba72-8a54a017d438 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:04.860632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a06bf-bf98-48ca-ba72-8a54a017d438 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:04.871717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43a06bf-bf98-48ca-ba72-8a54a017d438 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:04.877306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-b43a06bf-bf98-48ca-ba72-8a54a017d438 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:18:04.877306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:04.877306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:04.877306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:04.877306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:04.877306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:04.877306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:04.877306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:04.877306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:04.877306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:04.877306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:04.877306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:04.877306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:04.877306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:04.877306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:04.877306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:04.879472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:04.879472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:04.879472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:04.879472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:04.879472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:04.879472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:04.879472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:04.879472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:04.879472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:04.879472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:04.879472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:04.879472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:04.879472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:04.879472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:04.879472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:04.879472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:04.881379 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:04.881379 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:04.881379 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:04.881379 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:04.881379 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:04.881379 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:04.881379 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:04.881379 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:04.881379 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:04.881379 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:04.881379 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:18:04.881379 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:04.881379 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 235/479] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Sep 1 16:18:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:04.889128 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4c99e-e3ce-4292-822d-7aa8e1807e07 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:05.030952 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4c99e-e3ce-4292-822d-7aa8e1807e07 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:05.046903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:05.103608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e52d2-88a8-4dc6-8e0e-236087944ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:05.104146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e52d2-88a8-4dc6-8e0e-236087944ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:05.104426 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e52d2-88a8-4dc6-8e0e-236087944ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:05.104703 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e52d2-88a8-4dc6-8e0e-236087944ee2 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:05.105390 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e52d2-88a8-4dc6-8e0e-236087944ee2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:05.105682 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e52d2-88a8-4dc6-8e0e-236087944ee2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:05.107077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e52d2-88a8-4dc6-8e0e-236087944ee2 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:05.128592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 245/480] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:05.886473 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbd23f-86d5-4b58-8532-2a6febf32e6d None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:05.887549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbd23f-86d5-4b58-8532-2a6febf32e6d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:05.888207 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbd23f-86d5-4b58-8532-2a6febf32e6d None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:05.890271 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 236/481] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:05.895803 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e413e1-f162-4029-900b-790fd763515d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:05.896758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e413e1-f162-4029-900b-790fd763515d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:05.897262 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e413e1-f162-4029-900b-790fd763515d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:05.931417 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-93e413e1-f162-4029-900b-790fd763515d None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:05.932952 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-93e413e1-f162-4029-900b-790fd763515d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:06.023130 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 246/482] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:18:06.023431 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b163e4-3b41-4000-b3cb-d04077c29de4 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:06.025468 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b163e4-3b41-4000-b3cb-d04077c29de4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:06.025468 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b163e4-3b41-4000-b3cb-d04077c29de4 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:06.025468 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 237/483] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:06.030537 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293439d-1231-4764-bb33-b0fd0a43f381 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:06.031049 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293439d-1231-4764-bb33-b0fd0a43f381 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:06.031517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293439d-1231-4764-bb33-b0fd0a43f381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:06.059204 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e293439d-1231-4764-bb33-b0fd0a43f381 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:06.062224 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e293439d-1231-4764-bb33-b0fd0a43f381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:06.143401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 247/484] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:18:06.325105 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b163e4-3b41-4000-b3cb-d04077c29de4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:06.454865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b163e4-3b41-4000-b3cb-d04077c29de4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:06.506892 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b412ec1-9c10-4137-944b-d7d35506f50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:06.507528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b412ec1-9c10-4137-944b-d7d35506f50c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:06.507810 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b412ec1-9c10-4137-944b-d7d35506f50c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:06.508084 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b412ec1-9c10-4137-944b-d7d35506f50c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:06.523033 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b412ec1-9c10-4137-944b-d7d35506f50c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:06.528817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-9b412ec1-9c10-4137-944b-d7d35506f50c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:06.528817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:06.528817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:06.528817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:06.528817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:06.528817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:06.528817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:06.528817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:06.528817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:06.528817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:06.528817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:06.528817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:06.528817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:06.528817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:06.528817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:06.528817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:06.529715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:06.529715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:06.529715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:06.529715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:06.529715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:06.529715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:06.529715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:06.529715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:06.529715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:06.529715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:06.529715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:06.529715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:06.529715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:06.529715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:06.529715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:06.529715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:06.530575 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:06.530575 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:06.530575 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:06.530575 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:06.530575 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:06.530575 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:06.530575 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:06.530575 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:06.530575 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:06.530575 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:06.530575 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:06.530575 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:06.530575 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 238/485] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Sep 1 16:18:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:06.535920 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293439d-1231-4764-bb33-b0fd0a43f381 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:06.556381 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b412ec1-9c10-4137-944b-d7d35506f50c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:06.575854 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:06.578043 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:06.715615 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b412ec1-9c10-4137-944b-d7d35506f50c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:06.717691 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293439d-1231-4764-bb33-b0fd0a43f381 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:06.776946 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85817e-35e6-489b-bbbd-096c4184400f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:06.777942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85817e-35e6-489b-bbbd-096c4184400f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:06.778473 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85817e-35e6-489b-bbbd-096c4184400f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:06.778950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85817e-35e6-489b-bbbd-096c4184400f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:06.782328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cda20b-4de2-4337-9af3-ea669da58e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:06.783295 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cda20b-4de2-4337-9af3-ea669da58e36 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:06.784533 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cda20b-4de2-4337-9af3-ea669da58e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:06.787055 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cda20b-4de2-4337-9af3-ea669da58e36 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:06.787956 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cda20b-4de2-4337-9af3-ea669da58e36 None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:06.788464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cda20b-4de2-4337-9af3-ea669da58e36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:06.790664 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cda20b-4de2-4337-9af3-ea669da58e36 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:06.805720 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f85817e-35e6-489b-bbbd-096c4184400f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:06.811618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-2f85817e-35e6-489b-bbbd-096c4184400f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:06.811618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:06.811618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:06.811618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:06.811618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:06.811618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:06.811618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:06.811618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:06.811618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:06.811618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:06.811618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:06.811618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:06.811618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:06.811618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:06.811618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:06.811618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:06.812537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:06.812537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:06.812537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:06.812537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:06.812537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:06.812537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:06.812537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:06.812537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:06.812537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:06.812537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:06.812537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:06.812537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:06.812537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:06.812537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:06.812537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:06.812537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:06.813357 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:06.813357 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:06.813357 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:06.813357 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:06.813357 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:06.813357 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:06.813357 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:06.813357 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:06.813357 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:06.813357 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:06.813357 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:06.813357 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:06.814000 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 248/486] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Sep 1 16:18:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:06.815993 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 239/487] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Sep 1 16:18:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:18:06.826228 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cda20b-4de2-4337-9af3-ea669da58e36 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:06.827416 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85817e-35e6-489b-bbbd-096c4184400f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:06.918917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:06.924147 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:06.936745 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cda20b-4de2-4337-9af3-ea669da58e36 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:06.956197 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85817e-35e6-489b-bbbd-096c4184400f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:06.975143 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0329139b-f235-48e6-9564-4ed8e2f23c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:06.975724 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329139b-f235-48e6-9564-4ed8e2f23c61 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:06.976041 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329139b-f235-48e6-9564-4ed8e2f23c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:06.976327 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329139b-f235-48e6-9564-4ed8e2f23c61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:06.987779 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0329139b-f235-48e6-9564-4ed8e2f23c61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:06.989226 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0329139b-f235-48e6-9564-4ed8e2f23c61 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:06.994299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480c802-e380-4465-8e63-bdc6bd82e1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:06.995106 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480c802-e380-4465-8e63-bdc6bd82e1d8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:06.995572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480c802-e380-4465-8e63-bdc6bd82e1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:06.995843 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480c802-e380-4465-8e63-bdc6bd82e1d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:06.996514 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2480c802-e380-4465-8e63-bdc6bd82e1d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:06.996822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2480c802-e380-4465-8e63-bdc6bd82e1d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:06.998166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2480c802-e380-4465-8e63-bdc6bd82e1d8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:07.003587 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-0329139b-f235-48e6-9564-4ed8e2f23c61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:07.003587 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:07.003587 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:07.003587 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:07.003587 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:07.003587 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:07.003587 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:07.003587 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:07.003587 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:07.003587 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:07.003587 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:07.003587 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:07.003587 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:07.003587 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:07.003587 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:07.003587 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:07.004499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:07.004499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:07.004499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:07.004499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:07.004499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:07.004499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:07.004499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:07.004499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:07.004499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:07.004499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:07.004499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:07.004499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:07.004499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:07.004499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:07.004499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:07.004499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:07.004499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:07.010259 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:07.010259 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:07.010259 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:07.010259 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 249/488] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:07.010259 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44919f7a-4263-4f91-9c39-0356985f388a None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:07.010259 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44919f7a-4263-4f91-9c39-0356985f388a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:07.010259 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44919f7a-4263-4f91-9c39-0356985f388a None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:07.010259 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 250/489] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:07.010259 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4abc3-8961-467b-b6cd-08925f4b2765 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:07.010764 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4abc3-8961-467b-b6cd-08925f4b2765 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:07.010764 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4abc3-8961-467b-b6cd-08925f4b2765 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:07.014317 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 251/490] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:07.014660 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 240/491] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Sep 1 16:18:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:07.016108 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da11aa8-df12-4c57-bbae-d964ec5ed177 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:07.016397 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da11aa8-df12-4c57-bbae-d964ec5ed177 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:07.016671 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da11aa8-df12-4c57-bbae-d964ec5ed177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:07.019730 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480c802-e380-4465-8e63-bdc6bd82e1d8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:07.036623 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-0da11aa8-df12-4c57-bbae-d964ec5ed177 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:07.037808 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0da11aa8-df12-4c57-bbae-d964ec5ed177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:07.118646 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 252/492] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:18:07.120837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab0806-703c-4759-91c8-195b16fd97ce None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:07.121126 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab0806-703c-4759-91c8-195b16fd97ce None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:07.121402 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab0806-703c-4759-91c8-195b16fd97ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:07.130397 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480c802-e380-4465-8e63-bdc6bd82e1d8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:07.143285 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-6fab0806-703c-4759-91c8-195b16fd97ce None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:07.145002 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6fab0806-703c-4759-91c8-195b16fd97ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:07.162525 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:07.169243 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb019f-b501-4944-9c93-e102ede961df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:07.169853 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb019f-b501-4944-9c93-e102ede961df None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:07.170139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb019f-b501-4944-9c93-e102ede961df None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:07.170500 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb019f-b501-4944-9c93-e102ede961df None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:07.171152 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb019f-b501-4944-9c93-e102ede961df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:07.171461 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb019f-b501-4944-9c93-e102ede961df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:07.172848 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb019f-b501-4944-9c93-e102ede961df None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:07.189617 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 241/493] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:07.189617 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb019f-b501-4944-9c93-e102ede961df None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:07.245953 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 253/494] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:18:07.247279 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab0806-703c-4759-91c8-195b16fd97ce None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:07.249891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:07.360572 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab0806-703c-4759-91c8-195b16fd97ce None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:07.396315 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb019f-b501-4944-9c93-e102ede961df None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:07.402709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e7eee-449f-4f89-a0a2-fa903316ca92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:07.403536 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e7eee-449f-4f89-a0a2-fa903316ca92 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:07.403960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e7eee-449f-4f89-a0a2-fa903316ca92 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:07.404400 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e7eee-449f-4f89-a0a2-fa903316ca92 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:07.415334 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10e7eee-449f-4f89-a0a2-fa903316ca92 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:07.417866 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10e7eee-449f-4f89-a0a2-fa903316ca92 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:07.433602 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f048d-5e0b-44da-9d00-ab11e8387530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:07.434255 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f048d-5e0b-44da-9d00-ab11e8387530 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:07.434537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f048d-5e0b-44da-9d00-ab11e8387530 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:07.434813 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f048d-5e0b-44da-9d00-ab11e8387530 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:07.436861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-d10e7eee-449f-4f89-a0a2-fa903316ca92 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:18:07.436861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:07.436861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:07.436861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:07.436861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:07.436861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:07.436861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:07.436861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:07.436861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:07.436861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:07.436861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:07.436861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:07.436861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:07.436861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:07.436861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:07.436861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:07.437949 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:07.437949 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:07.437949 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:07.437949 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:07.437949 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:07.437949 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:07.437949 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:07.437949 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:07.437949 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:07.437949 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:07.437949 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:07.437949 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:07.437949 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:07.437949 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:07.437949 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:07.437949 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:07.437949 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:07.437949 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:07.439108 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:07.439108 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:07.439108 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:07.439108 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:07.439108 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:07.439108 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:07.439108 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:07.439108 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:07.439108 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:18:07.439108 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:07.446443 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 254/495] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Sep 1 16:18:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:07.447203 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e7eee-449f-4f89-a0a2-fa903316ca92 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:07.452419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f048d-5e0b-44da-9d00-ab11e8387530 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:07.454134 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f048d-5e0b-44da-9d00-ab11e8387530 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:07.475100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-2a5f048d-5e0b-44da-9d00-ab11e8387530 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:07.475100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:07.475100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:07.475100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:07.475100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:07.475100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:07.475100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:07.475100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:07.475100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:07.475100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:07.475100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:07.475100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:07.475100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:07.475100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:07.475100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:07.475100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:07.476163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:07.476163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:07.476163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:07.476163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:07.476163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:07.476163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:07.476163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:07.476163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:07.476163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:07.476163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:07.476163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:07.476163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:07.476163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:07.476163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:07.476163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:07.476163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:07.476163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:07.477105 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:07.477105 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:07.477105 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:07.479814 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 242/496] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:07.480913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f048d-5e0b-44da-9d00-ab11e8387530 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:07.527551 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:07.532627 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:07.583998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e7eee-449f-4f89-a0a2-fa903316ca92 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:07.592768 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f048d-5e0b-44da-9d00-ab11e8387530 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:07.619837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c6994-4de8-48d0-b380-830a6ff8bb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:07.620473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c6994-4de8-48d0-b380-830a6ff8bb30 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:07.620757 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c6994-4de8-48d0-b380-830a6ff8bb30 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:07.621032 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c6994-4de8-48d0-b380-830a6ff8bb30 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:07.627343 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d569e7-b30e-478a-9664-e0d58184b2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:07.627949 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d569e7-b30e-478a-9664-e0d58184b2f6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:07.628233 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d569e7-b30e-478a-9664-e0d58184b2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:07.628507 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d569e7-b30e-478a-9664-e0d58184b2f6 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:07.629249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d569e7-b30e-478a-9664-e0d58184b2f6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:07.629550 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d569e7-b30e-478a-9664-e0d58184b2f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:07.631013 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d569e7-b30e-478a-9664-e0d58184b2f6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:07.635154 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c6994-4de8-48d0-b380-830a6ff8bb30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:07.640602 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-e88c6994-4de8-48d0-b380-830a6ff8bb30 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:07.640602 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:07.640602 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:07.640602 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:07.640602 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:07.640602 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:07.640602 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:07.640602 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:07.640602 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:07.640602 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:07.640602 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:07.640602 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:07.640602 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:07.640602 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:07.640602 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:07.640602 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:07.641595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:07.641595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:07.641595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:07.641595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:07.641595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:07.641595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:07.641595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:07.641595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:07.641595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:07.641595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:07.641595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:07.641595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:07.641595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:07.641595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:07.641595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:07.641595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:07.647257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:07.647257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:07.647257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:07.647257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:07.647257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:07.647257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:07.647257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:07.647257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:07.647257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:07.647257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:07.647257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:07.647257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:07.647257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 255/497] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Sep 1 16:18:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:07.647257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c6994-4de8-48d0-b380-830a6ff8bb30 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:07.675700 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 243/498] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Sep 1 16:18:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:07.676573 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d569e7-b30e-478a-9664-e0d58184b2f6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:07.755592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c6994-4de8-48d0-b380-830a6ff8bb30 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:07.780402 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:07.788520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d569e7-b30e-478a-9664-e0d58184b2f6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:07.790829 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:07.796733 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea11d6a-414f-480a-a761-5d127ef0151e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:07.797609 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea11d6a-414f-480a-a761-5d127ef0151e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:07.797893 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea11d6a-414f-480a-a761-5d127ef0151e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:07.798174 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea11d6a-414f-480a-a761-5d127ef0151e None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:07.798853 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea11d6a-414f-480a-a761-5d127ef0151e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:07.799166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea11d6a-414f-480a-a761-5d127ef0151e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:07.800539 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea11d6a-414f-480a-a761-5d127ef0151e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:07.812915 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 256/499] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:07.817963 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea11d6a-414f-480a-a761-5d127ef0151e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:07.825667 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab226f7-c49e-4e3a-9f6b-c96b023fd0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:07.826339 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab226f7-c49e-4e3a-9f6b-c96b023fd0c5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:07.826624 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab226f7-c49e-4e3a-9f6b-c96b023fd0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:07.826898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab226f7-c49e-4e3a-9f6b-c96b023fd0c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:07.837646 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab226f7-c49e-4e3a-9f6b-c96b023fd0c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:07.842447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-2ab226f7-c49e-4e3a-9f6b-c96b023fd0c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:07.842447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:07.842447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:07.842447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:07.842447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:07.842447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:07.842447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:07.842447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:07.842447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:07.842447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:07.842447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:07.842447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:07.842447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:07.842447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:07.842447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:07.842447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:07.843392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:07.843392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:07.843392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:07.843392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:07.843392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:07.843392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:07.843392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:07.843392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:07.843392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:07.843392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:07.843392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:07.843392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:07.843392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:07.843392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:07.843392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:07.843392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:07.844219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:07.844219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:07.844219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:07.844219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:07.844219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:07.844219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:07.844219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:07.844219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:07.844219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:07.844219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:07.844219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:07.844219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:07.844219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 244/500] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Sep 1 16:18:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:07.844894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab226f7-c49e-4e3a-9f6b-c96b023fd0c5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:07.916886 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea11d6a-414f-480a-a761-5d127ef0151e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:07.943210 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab226f7-c49e-4e3a-9f6b-c96b023fd0c5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:07.950331 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6096ad59-8a99-46d3-a32d-0a41950f558c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:07.950933 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096ad59-8a99-46d3-a32d-0a41950f558c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:07.951213 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096ad59-8a99-46d3-a32d-0a41950f558c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:07.951485 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096ad59-8a99-46d3-a32d-0a41950f558c None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:07.952150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6096ad59-8a99-46d3-a32d-0a41950f558c None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:07.952450 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6096ad59-8a99-46d3-a32d-0a41950f558c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:07.953813 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6096ad59-8a99-46d3-a32d-0a41950f558c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:07.962120 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 257/501] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Sep 1 16:18:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:07.963442 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4443985-8930-4508-8188-341e9d14c9ea None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:07.963729 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4443985-8930-4508-8188-341e9d14c9ea None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:07.963997 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4443985-8930-4508-8188-341e9d14c9ea None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:07.965348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 258/502] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:07.965870 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4443985-8930-4508-8188-341e9d14c9ea None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:07.982157 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37bfa9c-a845-49af-a709-b0fd5e04e944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:07.982793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37bfa9c-a845-49af-a709-b0fd5e04e944 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:07.983084 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37bfa9c-a845-49af-a709-b0fd5e04e944 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:07.983365 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37bfa9c-a845-49af-a709-b0fd5e04e944 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:08.019948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:08.023110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:08.066906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37bfa9c-a845-49af-a709-b0fd5e04e944 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:08.068394 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37bfa9c-a845-49af-a709-b0fd5e04e944 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:08.085350 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4443985-8930-4508-8188-341e9d14c9ea None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:08.118344 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5385c8-eb92-4905-9d30-6fb1b6b822a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:08.118980 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5385c8-eb92-4905-9d30-6fb1b6b822a5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:08.119256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5385c8-eb92-4905-9d30-6fb1b6b822a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:08.119519 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5385c8-eb92-4905-9d30-6fb1b6b822a5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:08.126823 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5385c8-eb92-4905-9d30-6fb1b6b822a5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:08.128245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5385c8-eb92-4905-9d30-6fb1b6b822a5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:08.139345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-8f5385c8-eb92-4905-9d30-6fb1b6b822a5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 16:18:08.139345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:08.139345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:08.139345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:08.139345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:08.139345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:08.139345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:08.139345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:08.139345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:08.139345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:08.139345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:08.139345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:08.139345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:08.139345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:08.139345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:08.139345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:08.140453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:08.140453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:08.140453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:08.140453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:08.140453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:08.140453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:08.140453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:08.140453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:08.140453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:08.140453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:08.140453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:08.140453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:08.140453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:08.140453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:08.140453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:08.140453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:08.140453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:08.140453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:08.141389 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:08.141389 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:08.141389 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:08.141389 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:08.141389 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:08.141389 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:08.141389 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:08.141389 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:08.141389 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 16:18:08.141389 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:08.141389 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 259/503] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:18:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:08.141389 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5385c8-eb92-4905-9d30-6fb1b6b822a5 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:08.171188 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 245/504] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:18:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:18:08.172125 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37bfa9c-a845-49af-a709-b0fd5e04e944 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:08.231657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:08.245532 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5385c8-eb92-4905-9d30-6fb1b6b822a5 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:08.247856 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:08.271431 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37bfa9c-a845-49af-a709-b0fd5e04e944 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:08.280182 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb861f5-ab72-4695-b8e7-6e1bfc67c86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:08.280780 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb861f5-ab72-4695-b8e7-6e1bfc67c86a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:08.281065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb861f5-ab72-4695-b8e7-6e1bfc67c86a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:08.281337 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb861f5-ab72-4695-b8e7-6e1bfc67c86a None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:08.281996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb861f5-ab72-4695-b8e7-6e1bfc67c86a None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:08.282327 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb861f5-ab72-4695-b8e7-6e1bfc67c86a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:08.283687 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb861f5-ab72-4695-b8e7-6e1bfc67c86a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:08.292773 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 260/505] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Sep 1 16:18:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:08.294024 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc547e1-f021-4b96-8e79-21dfcf18323a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:08.294402 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc547e1-f021-4b96-8e79-21dfcf18323a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:08.294686 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc547e1-f021-4b96-8e79-21dfcf18323a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:08.307731 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd914f0-f746-4481-bc27-ad5af7200e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:08.308293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd914f0-f746-4481-bc27-ad5af7200e6c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:08.308576 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd914f0-f746-4481-bc27-ad5af7200e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:08.308854 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd914f0-f746-4481-bc27-ad5af7200e6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:08.311836 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ddc547e1-f021-4b96-8e79-21dfcf18323a None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:08.313034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ddc547e1-f021-4b96-8e79-21dfcf18323a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:08.318521 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd914f0-f746-4481-bc27-ad5af7200e6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:08.319955 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd914f0-f746-4481-bc27-ad5af7200e6c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:08.345991 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-cfd914f0-f746-4481-bc27-ad5af7200e6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:08.345991 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:08.345991 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:08.345991 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:08.345991 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:08.345991 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:08.345991 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:08.345991 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:08.345991 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:08.345991 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:08.345991 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:08.345991 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:08.345991 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:08.345991 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:08.345991 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:08.345991 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:08.347024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:08.347024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:08.347024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:08.347024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:08.347024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:08.347024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:08.347024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:08.347024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:08.347024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:08.347024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:08.347024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:08.347024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:08.347024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:08.347024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:08.347024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:08.347024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:08.347024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:08.347896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:08.347896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:08.347896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:08.347896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 246/506] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:08.348150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd914f0-f746-4481-bc27-ad5af7200e6c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:08.395409 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 261/507] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:18:08.396255 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc547e1-f021-4b96-8e79-21dfcf18323a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:08.459599 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd914f0-f746-4481-bc27-ad5af7200e6c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:08.474490 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:08.497234 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e17b79-9bc6-497f-974e-a2b431801938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:08.497847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e17b79-9bc6-497f-974e-a2b431801938 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:08.498132 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e17b79-9bc6-497f-974e-a2b431801938 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:08.498632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e17b79-9bc6-497f-974e-a2b431801938 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:08.499166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e17b79-9bc6-497f-974e-a2b431801938 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:08.499564 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e17b79-9bc6-497f-974e-a2b431801938 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:08.500928 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e17b79-9bc6-497f-974e-a2b431801938 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:08.502092 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:08.509707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc547e1-f021-4b96-8e79-21dfcf18323a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:08.518862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 247/508] 10.222.0.58 () {56 vars in 1137 bytes} [Thu Sep 1 16:18:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:08.519584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e17b79-9bc6-497f-974e-a2b431801938 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:08.551186 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe12ac-88ae-48a3-9788-bf4cf3fcd8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:08.551829 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe12ac-88ae-48a3-9788-bf4cf3fcd8ad None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:08.552110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe12ac-88ae-48a3-9788-bf4cf3fcd8ad None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:08.552398 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe12ac-88ae-48a3-9788-bf4cf3fcd8ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:08.562958 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe12ac-88ae-48a3-9788-bf4cf3fcd8ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:08.564348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe12ac-88ae-48a3-9788-bf4cf3fcd8ad None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:08.579555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-1dfe12ac-88ae-48a3-9788-bf4cf3fcd8ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:08.579555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:08.579555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:08.579555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:08.579555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:08.579555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:08.579555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:08.579555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:08.579555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:08.579555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:08.579555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:08.579555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:08.579555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:08.579555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:08.579555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:08.579555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:08.580462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:08.580462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:08.580462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:08.580462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:08.580462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:08.580462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:08.580462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:08.580462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:08.580462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:08.580462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:08.580462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:08.580462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:08.580462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:08.580462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:08.580462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:08.580462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:08.580462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:08.581355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:08.581355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:08.581355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:08.581355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 262/509] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:08.581355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe12ac-88ae-48a3-9788-bf4cf3fcd8ad None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:08.629822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e17b79-9bc6-497f-974e-a2b431801938 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:08.665975 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8210c-4339-4d69-8b58-5655cf15703e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:08.666623 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8210c-4339-4d69-8b58-5655cf15703e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:08.666899 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8210c-4339-4d69-8b58-5655cf15703e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:08.667423 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8210c-4339-4d69-8b58-5655cf15703e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:08.678350 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade8210c-4339-4d69-8b58-5655cf15703e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:08.681332 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade8210c-4339-4d69-8b58-5655cf15703e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:08.691362 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe12ac-88ae-48a3-9788-bf4cf3fcd8ad None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:08.703622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-ade8210c-4339-4d69-8b58-5655cf15703e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:08.703622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:08.703622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:08.703622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:08.703622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:08.703622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:08.703622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:08.703622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:08.703622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:08.703622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:08.703622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:08.703622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:08.703622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:08.703622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:08.703622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:08.703622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:08.704559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:08.704559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:08.704559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:08.704559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:08.704559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:08.704559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:08.704559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:08.704559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:08.704559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:08.704559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:08.704559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:08.704559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:08.704559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:08.704559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:08.704559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:08.704559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:08.704559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:08.705421 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:08.705421 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:08.705421 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:08.705421 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 248/510] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:08.706506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8210c-4339-4d69-8b58-5655cf15703e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:08.729536 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf21b4-044d-4900-9b10-dc433ca9b7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:08.730167 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf21b4-044d-4900-9b10-dc433ca9b7e9 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:08.730479 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf21b4-044d-4900-9b10-dc433ca9b7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:08.730758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf21b4-044d-4900-9b10-dc433ca9b7e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:08.731404 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf21b4-044d-4900-9b10-dc433ca9b7e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:08.731706 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf21b4-044d-4900-9b10-dc433ca9b7e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:08.733094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf21b4-044d-4900-9b10-dc433ca9b7e9 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:08.734628 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:08.745796 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 263/511] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:08.746641 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf21b4-044d-4900-9b10-dc433ca9b7e9 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:08.757872 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:08.810772 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8210c-4339-4d69-8b58-5655cf15703e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:08.845303 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05587c9d-5a04-4244-aafd-8154366a657c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:08.845825 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05587c9d-5a04-4244-aafd-8154366a657c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:08.846071 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf21b4-044d-4900-9b10-dc433ca9b7e9 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:08.846169 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05587c9d-5a04-4244-aafd-8154366a657c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:08.846394 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05587c9d-5a04-4244-aafd-8154366a657c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:08.879999 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6a703-151e-4e94-b53f-40bcb0a0f75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:08.880595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6a703-151e-4e94-b53f-40bcb0a0f75d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:08.880872 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6a703-151e-4e94-b53f-40bcb0a0f75d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:08.881143 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6a703-151e-4e94-b53f-40bcb0a0f75d None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:08.881782 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6a703-151e-4e94-b53f-40bcb0a0f75d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:08.882082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6a703-151e-4e94-b53f-40bcb0a0f75d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:08.884078 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6a703-151e-4e94-b53f-40bcb0a0f75d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:08.894955 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 264/512] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:08.895736 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6a703-151e-4e94-b53f-40bcb0a0f75d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:08.912203 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05587c9d-5a04-4244-aafd-8154366a657c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:08.913655 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05587c9d-5a04-4244-aafd-8154366a657c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:08.937552 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:08.960143 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:08.993040 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6a703-151e-4e94-b53f-40bcb0a0f75d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:09.012714 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 249/513] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:18:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:18:09.013561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05587c9d-5a04-4244-aafd-8154366a657c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:09.027104 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8cf4c-b67e-42fc-b413-b453979b54d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:09.027906 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8cf4c-b67e-42fc-b413-b453979b54d5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:09.028162 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8cf4c-b67e-42fc-b413-b453979b54d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:09.028286 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8cf4c-b67e-42fc-b413-b453979b54d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:09.045247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8cf4c-b67e-42fc-b413-b453979b54d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:09.046903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8cf4c-b67e-42fc-b413-b453979b54d5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:09.063247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-54a8cf4c-b67e-42fc-b413-b453979b54d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:09.063247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:09.063247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:09.063247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:09.063247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:09.063247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:09.063247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:09.063247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:09.063247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:09.063247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:09.063247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:09.063247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:09.063247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:09.063247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:09.063247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:09.063247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:09.064219 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:09.064219 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:09.064219 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:09.064219 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:09.064219 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:09.064219 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:09.064219 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:09.064219 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:09.064219 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:09.064219 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:09.064219 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:09.064219 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:09.064219 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:09.064219 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:09.064219 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:09.064219 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:09.064219 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:09.065944 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:09.065944 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:09.065944 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:09.065944 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 265/514] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:09.065944 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8cf4c-b67e-42fc-b413-b453979b54d5 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:09.120724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05587c9d-5a04-4244-aafd-8154366a657c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:09.143922 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:09.155015 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df9d04-6b64-49b3-a14f-09d4874b556f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:09.155614 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df9d04-6b64-49b3-a14f-09d4874b556f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:09.155906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df9d04-6b64-49b3-a14f-09d4874b556f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:09.156182 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df9d04-6b64-49b3-a14f-09d4874b556f None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:09.156832 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df9d04-6b64-49b3-a14f-09d4874b556f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:09.157139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df9d04-6b64-49b3-a14f-09d4874b556f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:09.158538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df9d04-6b64-49b3-a14f-09d4874b556f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:09.165081 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8cf4c-b67e-42fc-b413-b453979b54d5 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:09.169985 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 250/515] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:09.170728 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df9d04-6b64-49b3-a14f-09d4874b556f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:09.187461 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:09.201857 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b6205-27ba-4b34-b804-4a5970b15583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:09.202504 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b6205-27ba-4b34-b804-4a5970b15583 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:09.202784 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b6205-27ba-4b34-b804-4a5970b15583 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:09.203051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b6205-27ba-4b34-b804-4a5970b15583 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:09.210340 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b6205-27ba-4b34-b804-4a5970b15583 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:09.211823 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b6205-27ba-4b34-b804-4a5970b15583 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:09.222726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-1f7b6205-27ba-4b34-b804-4a5970b15583 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:18:09.222726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:09.222726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:09.222726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:09.222726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:09.222726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:09.222726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:09.222726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:09.222726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:09.222726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:09.222726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:09.222726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:09.222726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:09.222726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:09.222726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:09.222726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:09.223774 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:09.223774 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:09.223774 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:09.223774 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:09.223774 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:09.223774 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:09.223774 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:09.223774 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:09.223774 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:09.223774 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:09.223774 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:09.223774 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:09.223774 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:09.223774 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:09.223774 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:09.223774 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:09.223774 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:09.223774 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:09.224865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:09.224865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:09.224865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:09.224865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:09.224865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:09.224865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:09.224865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:09.224865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:09.224865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:18:09.224865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:09.224865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 266/516] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:18:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:09.224865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b6205-27ba-4b34-b804-4a5970b15583 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:09.270811 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df9d04-6b64-49b3-a14f-09d4874b556f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:09.304312 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e0682-1057-4b64-ad8c-1a92755d1100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:09.304943 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e0682-1057-4b64-ad8c-1a92755d1100 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:09.305831 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e0682-1057-4b64-ad8c-1a92755d1100 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:09.306127 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e0682-1057-4b64-ad8c-1a92755d1100 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:09.312665 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e0682-1057-4b64-ad8c-1a92755d1100 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:09.314126 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e0682-1057-4b64-ad8c-1a92755d1100 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:09.324283 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b6205-27ba-4b34-b804-4a5970b15583 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:09.326129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-089e0682-1057-4b64-ad8c-1a92755d1100 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:09.326129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:09.326129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:09.326129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:09.326129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:09.326129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:09.326129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:09.326129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:09.326129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:09.326129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:09.326129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:09.326129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:09.326129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:09.326129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:09.326129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:09.327008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:09.327008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:09.327008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:09.327008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:09.327008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:09.327008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:09.327008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:09.327008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:09.327008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:09.327008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:09.327008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:09.327008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:09.327008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:09.327008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:09.327008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:09.327008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:09.327008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:09.328150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:09.328150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:09.328150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:09.328150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:09.328150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:09.328150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:09.328150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:09.328150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:09.328150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 251/517] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:09.328150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e0682-1057-4b64-ad8c-1a92755d1100 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:09.363663 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489ba99-d39f-44e1-9164-1809573c4097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:09.364290 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489ba99-d39f-44e1-9164-1809573c4097 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:09.364569 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489ba99-d39f-44e1-9164-1809573c4097 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:09.364841 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489ba99-d39f-44e1-9164-1809573c4097 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:09.375957 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3489ba99-d39f-44e1-9164-1809573c4097 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:09.377333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3489ba99-d39f-44e1-9164-1809573c4097 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:09.380816 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:09.392431 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-3489ba99-d39f-44e1-9164-1809573c4097 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:09.392431 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:09.392431 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:09.392431 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:09.392431 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:09.392431 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:09.392431 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:09.392431 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:09.392431 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:09.392431 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:09.392431 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:09.392431 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:09.392431 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:09.392431 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:09.392431 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:09.392431 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:09.393406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:09.393406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:09.393406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:09.393406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:09.393406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:09.393406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:09.393406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:09.393406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:09.393406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:09.393406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:09.393406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:09.393406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:09.393406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:09.393406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:09.393406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:09.393406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:09.393406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:09.394309 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:09.394309 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:09.394309 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:09.394309 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 267/518] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:09.394943 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489ba99-d39f-44e1-9164-1809573c4097 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:09.428476 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:09.442686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e0682-1057-4b64-ad8c-1a92755d1100 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:09.476927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abbec6-e450-4c15-94a3-dd91e7f2b55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:09.477533 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abbec6-e450-4c15-94a3-dd91e7f2b55f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:09.477812 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abbec6-e450-4c15-94a3-dd91e7f2b55f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:09.478083 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abbec6-e450-4c15-94a3-dd91e7f2b55f None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:09.478793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9abbec6-e450-4c15-94a3-dd91e7f2b55f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:09.479100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9abbec6-e450-4c15-94a3-dd91e7f2b55f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:09.480481 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9abbec6-e450-4c15-94a3-dd91e7f2b55f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:09.491793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 252/519] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:09.492582 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abbec6-e450-4c15-94a3-dd91e7f2b55f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:09.501403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489ba99-d39f-44e1-9164-1809573c4097 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:09.538656 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa3eb7-00db-4440-80ce-4728043a470d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:09.539275 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa3eb7-00db-4440-80ce-4728043a470d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:09.539593 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa3eb7-00db-4440-80ce-4728043a470d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:09.539870 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa3eb7-00db-4440-80ce-4728043a470d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:09.547377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefa3eb7-00db-4440-80ce-4728043a470d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:09.548813 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefa3eb7-00db-4440-80ce-4728043a470d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:09.559693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-fefa3eb7-00db-4440-80ce-4728043a470d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 16:18:09.559693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:09.559693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:09.559693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:09.559693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:09.559693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:09.559693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:09.559693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:09.559693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:09.559693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:09.559693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:09.559693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:09.559693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:09.559693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:09.559693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:09.559693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:09.560660 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:09.560660 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:09.560660 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:09.560660 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:09.560660 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:09.560660 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:09.560660 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:09.560660 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:09.560660 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:09.560660 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:09.560660 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:09.560660 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:09.560660 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:09.560660 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:09.560660 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:09.560660 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:09.560660 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:09.560660 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:09.561579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:09.561579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:09.561579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:09.561579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:09.561579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:09.561579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:09.561579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:09.561579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:09.561579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 16:18:09.561579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:09.561579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 268/520] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Sep 1 16:18:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:09.561579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa3eb7-00db-4440-80ce-4728043a470d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:09.589225 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abbec6-e450-4c15-94a3-dd91e7f2b55f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:09.591559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:09.624358 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda661e-06cc-4903-a40a-869b8ea7fc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:09.624959 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda661e-06cc-4903-a40a-869b8ea7fc43 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:09.625241 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda661e-06cc-4903-a40a-869b8ea7fc43 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:09.625523 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda661e-06cc-4903-a40a-869b8ea7fc43 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:09.626343 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda661e-06cc-4903-a40a-869b8ea7fc43 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:09.626597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda661e-06cc-4903-a40a-869b8ea7fc43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:09.628057 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda661e-06cc-4903-a40a-869b8ea7fc43 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:09.644375 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 253/521] 10.222.0.58 () {56 vars in 1137 bytes} [Thu Sep 1 16:18:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:09.645164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda661e-06cc-4903-a40a-869b8ea7fc43 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:09.658794 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:09.668334 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa3eb7-00db-4440-80ce-4728043a470d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:09.703914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36d9df-41a1-43d5-b4a5-c79a5bb7dcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:09.704520 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36d9df-41a1-43d5-b4a5-c79a5bb7dcbe None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:09.704801 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36d9df-41a1-43d5-b4a5-c79a5bb7dcbe None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:09.705075 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36d9df-41a1-43d5-b4a5-c79a5bb7dcbe None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:09.705771 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff36d9df-41a1-43d5-b4a5-c79a5bb7dcbe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:09.706076 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff36d9df-41a1-43d5-b4a5-c79a5bb7dcbe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:09.707541 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff36d9df-41a1-43d5-b4a5-c79a5bb7dcbe None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:09.725702 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 269/522] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:09.726401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36d9df-41a1-43d5-b4a5-c79a5bb7dcbe None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:09.756506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda661e-06cc-4903-a40a-869b8ea7fc43 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:09.791519 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c29a52-2912-4cf6-aaca-d9dc375406ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:09.792575 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c29a52-2912-4cf6-aaca-d9dc375406ab None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:09.792689 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c29a52-2912-4cf6-aaca-d9dc375406ab None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:09.792923 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c29a52-2912-4cf6-aaca-d9dc375406ab None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:09.793566 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c29a52-2912-4cf6-aaca-d9dc375406ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:09.793870 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c29a52-2912-4cf6-aaca-d9dc375406ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:09.795277 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c29a52-2912-4cf6-aaca-d9dc375406ab None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:09.806362 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 254/523] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:09.807141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c29a52-2912-4cf6-aaca-d9dc375406ab None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:09.828099 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36d9df-41a1-43d5-b4a5-c79a5bb7dcbe None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:09.830186 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:09.862568 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4333bdb9-8e8c-46da-8c96-1c350efd2328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:09.863174 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4333bdb9-8e8c-46da-8c96-1c350efd2328 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:09.863453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4333bdb9-8e8c-46da-8c96-1c350efd2328 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:09.863724 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4333bdb9-8e8c-46da-8c96-1c350efd2328 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:09.864364 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4333bdb9-8e8c-46da-8c96-1c350efd2328 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:09.865781 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4333bdb9-8e8c-46da-8c96-1c350efd2328 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:09.886637 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 270/524] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Sep 1 16:18:09 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:18:09.887371 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4333bdb9-8e8c-46da-8c96-1c350efd2328 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:09.899922 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:09.923207 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c29a52-2912-4cf6-aaca-d9dc375406ab None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:09.963129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf923140-6332-4946-a718-efe9646b5ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:09.963746 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf923140-6332-4946-a718-efe9646b5ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:09.964024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf923140-6332-4946-a718-efe9646b5ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:09.964296 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf923140-6332-4946-a718-efe9646b5ea8 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:09.965052 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf923140-6332-4946-a718-efe9646b5ea8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:09.965357 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf923140-6332-4946-a718-efe9646b5ea8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:09.966851 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf923140-6332-4946-a718-efe9646b5ea8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:09.993465 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 255/525] 10.222.0.58 () {56 vars in 1139 bytes} [Thu Sep 1 16:18:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:09.994397 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf923140-6332-4946-a718-efe9646b5ea8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:10.010462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4333bdb9-8e8c-46da-8c96-1c350efd2328 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:10.055393 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ff0b2-2cbf-4605-ae6a-2dcd42127efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:10.055998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ff0b2-2cbf-4605-ae6a-2dcd42127efd None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:10.056273 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ff0b2-2cbf-4605-ae6a-2dcd42127efd None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:10.056540 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ff0b2-2cbf-4605-ae6a-2dcd42127efd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:10.096748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:10.118549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf923140-6332-4946-a718-efe9646b5ea8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:10.136722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751ff0b2-2cbf-4605-ae6a-2dcd42127efd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:10.138117 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751ff0b2-2cbf-4605-ae6a-2dcd42127efd None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:10.149032 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:10.158151 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a2da8-c06a-4505-8429-6c78022046c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:10.158797 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a2da8-c06a-4505-8429-6c78022046c3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:10.159100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a2da8-c06a-4505-8429-6c78022046c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:10.159375 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a2da8-c06a-4505-8429-6c78022046c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:10.160098 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a2da8-c06a-4505-8429-6c78022046c3 None admin] RBAC: Adding query filter params (role.id=b0923c7a240149ae9ece5b3c1622e860, scope.project.id=479530ced580426b85d1fc07af43417f, user.id=1f0c99fdf8bd4331ad7e641ee38554c1) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:10.160404 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a2da8-c06a-4505-8429-6c78022046c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:10.161837 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a2da8-c06a-4505-8429-6c78022046c3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:10.168144 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 256/526] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:18:09 2022] GET /identity/v3/role_assignments?user.id=1f0c99fdf8bd4331ad7e641ee38554c1&scope.project.id=479530ced580426b85d1fc07af43417f&role.id=b0923c7a240149ae9ece5b3c1622e860 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:10.168921 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a2da8-c06a-4505-8429-6c78022046c3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:10.290518 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 271/527] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:18:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:18:10.291302 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ff0b2-2cbf-4605-ae6a-2dcd42127efd None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:10.300963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a2da8-c06a-4505-8429-6c78022046c3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:10.341738 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ce5f8-3b8c-41c5-991d-5cb89ab1def4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:10.344630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ce5f8-3b8c-41c5-991d-5cb89ab1def4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:10.344630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ce5f8-3b8c-41c5-991d-5cb89ab1def4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:10.344630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ce5f8-3b8c-41c5-991d-5cb89ab1def4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:10.351076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9ce5f8-3b8c-41c5-991d-5cb89ab1def4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:10.353121 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9ce5f8-3b8c-41c5-991d-5cb89ab1def4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:10.371249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-1b9ce5f8-3b8c-41c5-991d-5cb89ab1def4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:10.371249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:10.371249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:10.371249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:10.371249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:10.371249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:10.371249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:10.371249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:10.371249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:10.371249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:10.371249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:10.371249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:10.371249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:10.371249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:10.371249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:10.372181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:10.372181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:10.372181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:10.372181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:10.372181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:10.372181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:10.372181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:10.372181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:10.372181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:10.372181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:10.372181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:10.372181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:10.372181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:10.372181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:10.372181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:10.372181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:10.372181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:10.373042 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:10.373042 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:10.373042 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:10.373042 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:10.373042 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:10.373042 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:10.373042 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:10.373042 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:10.380688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 257/528] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:10.380688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ce5f8-3b8c-41c5-991d-5cb89ab1def4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:10.385758 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:10.434344 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ff0b2-2cbf-4605-ae6a-2dcd42127efd None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:10.446533 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:10.471227 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39fb77e-8827-4b60-ad7f-872a24c0be7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:10.472062 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fb77e-8827-4b60-ad7f-872a24c0be7f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:10.472438 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fb77e-8827-4b60-ad7f-872a24c0be7f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:10.472826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fb77e-8827-4b60-ad7f-872a24c0be7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:10.504646 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ce5f8-3b8c-41c5-991d-5cb89ab1def4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:10.552827 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d637df8-d259-4210-9138-8a5a532e2317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:10.553234 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39fb77e-8827-4b60-ad7f-872a24c0be7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:10.553851 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d637df8-d259-4210-9138-8a5a532e2317 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:10.554299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d637df8-d259-4210-9138-8a5a532e2317 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:10.554736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d637df8-d259-4210-9138-8a5a532e2317 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:10.555695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39fb77e-8827-4b60-ad7f-872a24c0be7f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:10.568882 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d637df8-d259-4210-9138-8a5a532e2317 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:10.571538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d637df8-d259-4210-9138-8a5a532e2317 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:10.591164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-4d637df8-d259-4210-9138-8a5a532e2317 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:10.591164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:10.591164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:10.591164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:10.591164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:10.591164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:10.591164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:10.591164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:10.591164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:10.591164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:10.591164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:10.591164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:10.591164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:10.591164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:10.591164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:10.591164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:10.592198 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:10.592198 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:10.592198 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:10.592198 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:10.592198 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:10.592198 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:10.592198 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:10.592198 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:10.592198 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:10.592198 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:10.592198 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:10.592198 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:10.592198 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:10.592198 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:10.592198 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:10.592198 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:10.592198 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:10.593066 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:10.593066 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:10.593066 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:10.595869 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 258/529] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:10.596812 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d637df8-d259-4210-9138-8a5a532e2317 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:10.669262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:10.669918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 272/530] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:18:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:18:10.671051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39fb77e-8827-4b60-ad7f-872a24c0be7f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:10.688419 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:10.739045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d637df8-d259-4210-9138-8a5a532e2317 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:10.783948 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39fb77e-8827-4b60-ad7f-872a24c0be7f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:10.786541 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bdcb03-ea06-447f-8ee9-a6c8c18c2b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:10.788231 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bdcb03-ea06-447f-8ee9-a6c8c18c2b81 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:10.788666 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bdcb03-ea06-447f-8ee9-a6c8c18c2b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:10.789092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bdcb03-ea06-447f-8ee9-a6c8c18c2b81 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:10.790527 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bdcb03-ea06-447f-8ee9-a6c8c18c2b81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:10.790986 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bdcb03-ea06-447f-8ee9-a6c8c18c2b81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:10.792839 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bdcb03-ea06-447f-8ee9-a6c8c18c2b81 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:10.822230 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 259/531] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:10.822230 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bdcb03-ea06-447f-8ee9-a6c8c18c2b81 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:10.833439 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e486aa-95d2-47f1-a9b5-52e72735fd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:10.834792 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e486aa-95d2-47f1-a9b5-52e72735fd25 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:10.835560 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e486aa-95d2-47f1-a9b5-52e72735fd25 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:10.836061 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e486aa-95d2-47f1-a9b5-52e72735fd25 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:10.837173 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e486aa-95d2-47f1-a9b5-52e72735fd25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:10.838165 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e486aa-95d2-47f1-a9b5-52e72735fd25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:10.840412 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e486aa-95d2-47f1-a9b5-52e72735fd25 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:10.859243 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 273/532] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:10.860788 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e486aa-95d2-47f1-a9b5-52e72735fd25 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:10.931373 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bdcb03-ea06-447f-8ee9-a6c8c18c2b81 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:10.953571 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:10.955556 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:10.976939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7118807f-78aa-4cb5-bc9e-cad616cd44c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:10.977912 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118807f-78aa-4cb5-bc9e-cad616cd44c1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:10.978425 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118807f-78aa-4cb5-bc9e-cad616cd44c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:10.978902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118807f-78aa-4cb5-bc9e-cad616cd44c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:10.989451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7118807f-78aa-4cb5-bc9e-cad616cd44c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:10.991176 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7118807f-78aa-4cb5-bc9e-cad616cd44c1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:10.992008 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e486aa-95d2-47f1-a9b5-52e72735fd25 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:11.007708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-7118807f-78aa-4cb5-bc9e-cad616cd44c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:11.007708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:11.007708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:11.007708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:11.007708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:11.007708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:11.007708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:11.007708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:11.007708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:11.007708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:11.007708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:11.007708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:11.007708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:11.007708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:11.007708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:11.007708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:11.009184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:11.009184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:11.009184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:11.009184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:11.009184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:11.009184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:11.009184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:11.009184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:11.009184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:11.009184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:11.009184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:11.009184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:11.009184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:11.009184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:11.009184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:11.009184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:11.009184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:11.010050 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:11.010050 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:11.010050 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:11.016350 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 260/533] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:11.017550 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7118807f-78aa-4cb5-bc9e-cad616cd44c1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:11.029613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37014197-0644-45e2-b847-4dc92d6dc887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:11.030235 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37014197-0644-45e2-b847-4dc92d6dc887 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:11.030235 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37014197-0644-45e2-b847-4dc92d6dc887 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:11.030376 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37014197-0644-45e2-b847-4dc92d6dc887 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:11.031066 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37014197-0644-45e2-b847-4dc92d6dc887 None admin] RBAC: Adding query filter params (role.id=b0923c7a240149ae9ece5b3c1622e860, scope.project.id=479530ced580426b85d1fc07af43417f, user.id=6a2c872129414105a112661cc2b477e6) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:11.031439 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37014197-0644-45e2-b847-4dc92d6dc887 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:11.032958 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37014197-0644-45e2-b847-4dc92d6dc887 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:11.046238 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 274/534] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:18:10 2022] GET /identity/v3/role_assignments?user.id=6a2c872129414105a112661cc2b477e6&scope.project.id=479530ced580426b85d1fc07af43417f&role.id=b0923c7a240149ae9ece5b3c1622e860 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:11.046238 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37014197-0644-45e2-b847-4dc92d6dc887 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:11.190014 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37014197-0644-45e2-b847-4dc92d6dc887 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:11.236417 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7118807f-78aa-4cb5-bc9e-cad616cd44c1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:11.247332 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:11.254218 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616e7a2-1c49-42fe-8c7b-26572794aa6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:11.255113 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616e7a2-1c49-42fe-8c7b-26572794aa6c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:11.255757 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616e7a2-1c49-42fe-8c7b-26572794aa6c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:11.256437 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616e7a2-1c49-42fe-8c7b-26572794aa6c None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:11.258934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616e7a2-1c49-42fe-8c7b-26572794aa6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:11.259581 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616e7a2-1c49-42fe-8c7b-26572794aa6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:11.261361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616e7a2-1c49-42fe-8c7b-26572794aa6c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:11.291548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 275/535] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:11.294535 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616e7a2-1c49-42fe-8c7b-26572794aa6c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:11.310060 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58e527-f20a-4f82-a50e-5e0baa93936c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:11.311082 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58e527-f20a-4f82-a50e-5e0baa93936c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:11.311578 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58e527-f20a-4f82-a50e-5e0baa93936c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:11.312047 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58e527-f20a-4f82-a50e-5e0baa93936c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:11.323499 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff58e527-f20a-4f82-a50e-5e0baa93936c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:11.326280 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff58e527-f20a-4f82-a50e-5e0baa93936c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:11.349076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-ff58e527-f20a-4f82-a50e-5e0baa93936c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:11.349076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:11.349076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:11.349076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:11.349076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:11.349076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:11.349076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:11.349076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:11.349076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:11.349076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:11.349076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:11.349076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:11.349076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:11.349076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:11.349076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:11.350017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:11.350017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:11.350017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:11.350017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:11.350017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:11.350017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:11.350017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:11.350017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:11.350017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:11.350017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:11.350017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:11.350017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:11.350017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:11.350017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:11.350017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:11.350017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:11.350017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:11.350932 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:11.350932 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:11.350932 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:11.350932 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:11.350932 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:11.350932 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:11.350932 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:11.350932 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:11.359700 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 261/536] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:11.361829 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58e527-f20a-4f82-a50e-5e0baa93936c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:11.423558 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:11.458130 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616e7a2-1c49-42fe-8c7b-26572794aa6c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:11.519741 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb057cd-836c-4d5b-a54f-a4519a9930d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:11.522988 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb057cd-836c-4d5b-a54f-a4519a9930d5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:11.523486 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb057cd-836c-4d5b-a54f-a4519a9930d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:11.523993 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb057cd-836c-4d5b-a54f-a4519a9930d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:11.536444 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb057cd-836c-4d5b-a54f-a4519a9930d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:11.538972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb057cd-836c-4d5b-a54f-a4519a9930d5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:11.554239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-2eb057cd-836c-4d5b-a54f-a4519a9930d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:11.554239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:11.554239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:11.554239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:11.554239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:11.554239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:11.554239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:11.554239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:11.554239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:11.554239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:11.554239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:11.554239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:11.554239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:11.554239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:11.554239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:11.555712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:11.555712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:11.555712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:11.555712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:11.555712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:11.555712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:11.555712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:11.555712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:11.555712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:11.555712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:11.555712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:11.555712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:11.555712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:11.555712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:11.555712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:11.555712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:11.555712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:11.556668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:11.556668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:11.556668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:11.556668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:11.556668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:11.556668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:11.556668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:11.556668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:11.563416 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 276/537] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:11.564972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb057cd-836c-4d5b-a54f-a4519a9930d5 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:11.568421 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58e527-f20a-4f82-a50e-5e0baa93936c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:11.640105 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d78c5-675b-4cfd-9875-d235e81aeea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:11.641133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d78c5-675b-4cfd-9875-d235e81aeea1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:11.643891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d78c5-675b-4cfd-9875-d235e81aeea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:11.644429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d78c5-675b-4cfd-9875-d235e81aeea1 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:11.645338 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d78c5-675b-4cfd-9875-d235e81aeea1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:11.645935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d78c5-675b-4cfd-9875-d235e81aeea1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:11.649958 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d78c5-675b-4cfd-9875-d235e81aeea1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:11.668559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:11.678956 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 262/538] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:11.687232 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d78c5-675b-4cfd-9875-d235e81aeea1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:11.741939 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb057cd-836c-4d5b-a54f-a4519a9930d5 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:11.816061 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970d2bb-aa40-4a22-bb6a-f16e9a15bec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:11.816598 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970d2bb-aa40-4a22-bb6a-f16e9a15bec5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:11.816879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970d2bb-aa40-4a22-bb6a-f16e9a15bec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:11.817152 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970d2bb-aa40-4a22-bb6a-f16e9a15bec5 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:11.817978 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970d2bb-aa40-4a22-bb6a-f16e9a15bec5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:11.818281 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970d2bb-aa40-4a22-bb6a-f16e9a15bec5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:11.819850 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970d2bb-aa40-4a22-bb6a-f16e9a15bec5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:11.830918 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:11.851739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 277/539] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:11.857567 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d78c5-675b-4cfd-9875-d235e81aeea1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:11.868242 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970d2bb-aa40-4a22-bb6a-f16e9a15bec5 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:11.917649 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386e21c3-b99d-439c-9c79-ce15fda7e391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:11.918329 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e21c3-b99d-439c-9c79-ce15fda7e391 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:11.918617 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e21c3-b99d-439c-9c79-ce15fda7e391 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:11.918895 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e21c3-b99d-439c-9c79-ce15fda7e391 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:11.919627 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386e21c3-b99d-439c-9c79-ce15fda7e391 None admin] RBAC: Adding query filter params (role.id=b0923c7a240149ae9ece5b3c1622e860, scope.project.id=479530ced580426b85d1fc07af43417f, user.id=84d4bd0a456341f78b5e476c97a9165c) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:11.919942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386e21c3-b99d-439c-9c79-ce15fda7e391 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:11.921386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386e21c3-b99d-439c-9c79-ce15fda7e391 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:11.932459 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 263/540] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:18:11 2022] GET /identity/v3/role_assignments?user.id=84d4bd0a456341f78b5e476c97a9165c&scope.project.id=479530ced580426b85d1fc07af43417f&role.id=b0923c7a240149ae9ece5b3c1622e860 => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:11.933920 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578a488-43d6-4c5a-9c1a-86ca8c05fb4a None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:11.934232 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578a488-43d6-4c5a-9c1a-86ca8c05fb4a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:11.934507 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578a488-43d6-4c5a-9c1a-86ca8c05fb4a None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:11.936156 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 264/541] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:11.941366 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c6563-b0fd-4361-bda1-862dfed08e7a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:11.941653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c6563-b0fd-4361-bda1-862dfed08e7a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:11.941933 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c6563-b0fd-4361-bda1-862dfed08e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:11.974755 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ab6c6563-b0fd-4361-bda1-862dfed08e7a None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:11.975998 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ab6c6563-b0fd-4361-bda1-862dfed08e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:12.013423 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970d2bb-aa40-4a22-bb6a-f16e9a15bec5 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:12.040454 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:12.090570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e40c5a-4d14-4a2b-a448-5085e9c297c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:12.091107 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e40c5a-4d14-4a2b-a448-5085e9c297c3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:12.091385 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e40c5a-4d14-4a2b-a448-5085e9c297c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:12.091657 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e40c5a-4d14-4a2b-a448-5085e9c297c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:12.092406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e40c5a-4d14-4a2b-a448-5085e9c297c3 None admin] RBAC: Adding query filter params (role.id=b0923c7a240149ae9ece5b3c1622e860, scope.project.id=479530ced580426b85d1fc07af43417f, user.id=943fae29c5dc41f2911bbe051c74a361) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:12.092718 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e40c5a-4d14-4a2b-a448-5085e9c297c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:12.094179 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e40c5a-4d14-4a2b-a448-5085e9c297c3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:12.106422 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 278/542] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:18:11 2022] GET /identity/v3/role_assignments?user.id=943fae29c5dc41f2911bbe051c74a361&scope.project.id=479530ced580426b85d1fc07af43417f&role.id=b0923c7a240149ae9ece5b3c1622e860 => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:12.127288 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 265/543] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:18:12.237845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82666a57-7f03-48fd-ace0-f9b21defec13 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:12.238135 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82666a57-7f03-48fd-ace0-f9b21defec13 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:12.238432 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82666a57-7f03-48fd-ace0-f9b21defec13 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:12.239799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 279/544] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:12.247315 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59305241-8efa-4fd7-a08c-edcd767f83bf None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:12.247818 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59305241-8efa-4fd7-a08c-edcd767f83bf None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:12.247893 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59305241-8efa-4fd7-a08c-edcd767f83bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:12.267761 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-59305241-8efa-4fd7-a08c-edcd767f83bf None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:12.268960 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-59305241-8efa-4fd7-a08c-edcd767f83bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:12.350508 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 266/545] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:18:12.490216 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82666a57-7f03-48fd-ace0-f9b21defec13 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:12.618288 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82666a57-7f03-48fd-ace0-f9b21defec13 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:12.652422 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889740b-1082-49c1-837a-e0c2b3471d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:12.653051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889740b-1082-49c1-837a-e0c2b3471d30 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:12.653330 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889740b-1082-49c1-837a-e0c2b3471d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:12.653600 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889740b-1082-49c1-837a-e0c2b3471d30 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:12.664524 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889740b-1082-49c1-837a-e0c2b3471d30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:12.670220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-b889740b-1082-49c1-837a-e0c2b3471d30 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:12.670220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:12.670220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:12.670220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:12.670220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:12.670220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:12.670220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:12.670220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:12.670220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:12.670220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:12.670220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:12.670220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:12.670220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:12.670220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:12.670220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:12.670220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:12.671121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:12.671121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:12.671121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:12.671121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:12.671121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:12.671121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:12.671121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:12.671121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:12.671121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:12.671121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:12.671121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:12.671121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:12.671121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:12.671121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:12.671121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:12.671121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:12.671930 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:12.671930 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:12.671930 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:12.671930 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:12.671930 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:12.671930 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:12.671930 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:12.671930 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:12.671930 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:12.671930 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:12.671930 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:12.671930 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:12.671930 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 280/546] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Sep 1 16:18:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:12.677550 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59305241-8efa-4fd7-a08c-edcd767f83bf None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:12.693367 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:12.723097 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889740b-1082-49c1-837a-e0c2b3471d30 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:12.773042 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:12.844418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59305241-8efa-4fd7-a08c-edcd767f83bf None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:12.871074 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889740b-1082-49c1-837a-e0c2b3471d30 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:12.898658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c8f807-66ca-48b9-88ed-f2a41863739e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:12.899411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8f807-66ca-48b9-88ed-f2a41863739e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:12.899698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8f807-66ca-48b9-88ed-f2a41863739e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:12.899974 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8f807-66ca-48b9-88ed-f2a41863739e None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:12.900651 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c8f807-66ca-48b9-88ed-f2a41863739e None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:12.900954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c8f807-66ca-48b9-88ed-f2a41863739e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:12.902474 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c8f807-66ca-48b9-88ed-f2a41863739e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:12.907619 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4bcac0-80ae-4553-b1a0-8a99e54d3a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:12.908412 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4bcac0-80ae-4553-b1a0-8a99e54d3a62 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:12.908787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4bcac0-80ae-4553-b1a0-8a99e54d3a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:12.909155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4bcac0-80ae-4553-b1a0-8a99e54d3a62 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:12.911746 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 267/547] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Sep 1 16:18:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:12.917846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c8f807-66ca-48b9-88ed-f2a41863739e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:12.920653 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4bcac0-80ae-4553-b1a0-8a99e54d3a62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:12.926254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-6b4bcac0-80ae-4553-b1a0-8a99e54d3a62 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:12.926254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:12.926254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:12.926254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:12.926254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:12.926254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:12.926254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:12.926254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:12.926254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:12.926254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:12.926254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:12.926254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:12.926254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:12.926254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:12.926254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:12.926254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:12.927161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:12.927161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:12.927161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:12.927161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:12.927161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:12.927161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:12.927161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:12.927161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:12.927161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:12.927161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:12.927161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:12.927161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:12.927161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:12.927161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:12.927161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:12.927161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:12.927972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:12.927972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:12.927972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:12.927972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:12.927972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:12.927972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:12.927972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:12.927972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:12.927972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:12.927972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:12.927972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:12.927972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:12.927972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 281/548] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Sep 1 16:18:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:18:12.937388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4bcac0-80ae-4553-b1a0-8a99e54d3a62 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:13.014281 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:13.083783 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:13.088891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c8f807-66ca-48b9-88ed-f2a41863739e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:13.093722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4bcac0-80ae-4553-b1a0-8a99e54d3a62 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:13.135588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ac316-7fd9-44d1-8f39-c595128cfcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:13.136546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ac316-7fd9-44d1-8f39-c595128cfcee None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:13.137027 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ac316-7fd9-44d1-8f39-c595128cfcee None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:13.137543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ac316-7fd9-44d1-8f39-c595128cfcee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:13.151749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16ac316-7fd9-44d1-8f39-c595128cfcee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:13.153436 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16ac316-7fd9-44d1-8f39-c595128cfcee None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:13.169572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-e16ac316-7fd9-44d1-8f39-c595128cfcee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:13.169572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:13.169572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:13.169572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:13.169572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:13.169572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:13.169572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:13.169572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:13.169572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:13.169572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:13.169572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:13.169572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:13.169572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:13.169572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:13.169572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:13.169572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:13.171097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:13.171097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:13.171097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:13.171097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:13.171097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:13.171097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:13.171097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:13.171097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:13.171097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:13.171097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:13.171097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:13.171097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:13.171097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:13.171097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:13.171097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:13.171097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:13.171097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:13.172154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:13.172154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:13.172154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:13.172358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae4a91-50b3-420d-9e29-7e4d04cdcb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:13.172358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae4a91-50b3-420d-9e29-7e4d04cdcb46 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:13.172358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae4a91-50b3-420d-9e29-7e4d04cdcb46 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:13.172358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae4a91-50b3-420d-9e29-7e4d04cdcb46 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:13.173155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ae4a91-50b3-420d-9e29-7e4d04cdcb46 None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:13.173155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ae4a91-50b3-420d-9e29-7e4d04cdcb46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:13.174386 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ae4a91-50b3-420d-9e29-7e4d04cdcb46 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:13.175159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 268/549] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:13.181205 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ac316-7fd9-44d1-8f39-c595128cfcee None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:13.189756 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 282/550] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Sep 1 16:18:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:13.194982 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae4a91-50b3-420d-9e29-7e4d04cdcb46 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:13.321955 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae4a91-50b3-420d-9e29-7e4d04cdcb46 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:13.337082 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ac316-7fd9-44d1-8f39-c595128cfcee None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:13.363413 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:13.376503 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843e269-5749-4ecd-93b2-2c927c966430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:13.377149 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843e269-5749-4ecd-93b2-2c927c966430 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:13.377429 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843e269-5749-4ecd-93b2-2c927c966430 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:13.377704 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843e269-5749-4ecd-93b2-2c927c966430 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:13.379478 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431bda0-83e5-4fea-9ff8-91bc919bf261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:13.380082 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431bda0-83e5-4fea-9ff8-91bc919bf261 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:13.380362 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431bda0-83e5-4fea-9ff8-91bc919bf261 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:13.380636 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431bda0-83e5-4fea-9ff8-91bc919bf261 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:13.381296 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431bda0-83e5-4fea-9ff8-91bc919bf261 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:13.381604 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431bda0-83e5-4fea-9ff8-91bc919bf261 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:13.383013 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431bda0-83e5-4fea-9ff8-91bc919bf261 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:13.396341 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843e269-5749-4ecd-93b2-2c927c966430 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:13.396992 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 269/551] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:13.398334 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb88b45-7aef-4326-a149-c2953fad6459 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:13.398628 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb88b45-7aef-4326-a149-c2953fad6459 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:13.398765 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843e269-5749-4ecd-93b2-2c927c966430 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:13.398932 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb88b45-7aef-4326-a149-c2953fad6459 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:13.400405 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 270/552] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:13.402333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb88b45-7aef-4326-a149-c2953fad6459 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:13.418047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-8843e269-5749-4ecd-93b2-2c927c966430 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:13.418047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:13.418047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:13.418047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:13.418047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:13.418047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:13.418047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:13.418047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:13.418047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:13.418047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:13.418047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:13.418047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:13.418047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:13.418047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:13.418047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:13.418047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:13.419358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:13.419358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:13.419358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:13.419358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:13.419358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:13.419358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:13.419358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:13.419358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:13.419358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:13.419358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:13.419358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:13.419358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:13.419358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:13.419358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:13.419358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:13.419358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:13.419358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:13.420371 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:13.420371 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:13.420371 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:13.422926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 283/553] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:13.427718 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bcdb4b-6432-4b69-bbfa-016867ef8f5d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:13.427718 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bcdb4b-6432-4b69-bbfa-016867ef8f5d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:13.427900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bcdb4b-6432-4b69-bbfa-016867ef8f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:13.447710 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-25bcdb4b-6432-4b69-bbfa-016867ef8f5d None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:13.448902 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-25bcdb4b-6432-4b69-bbfa-016867ef8f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:13.451860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:13.526243 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 284/554] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:18:13.527302 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bcdb4b-6432-4b69-bbfa-016867ef8f5d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:13.546604 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb88b45-7aef-4326-a149-c2953fad6459 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:13.587332 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915e639-8ed2-4c3d-9f85-7b83e88fab73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:13.588280 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915e639-8ed2-4c3d-9f85-7b83e88fab73 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:13.588608 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915e639-8ed2-4c3d-9f85-7b83e88fab73 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:13.589206 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915e639-8ed2-4c3d-9f85-7b83e88fab73 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:13.596906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915e639-8ed2-4c3d-9f85-7b83e88fab73 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:13.598496 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915e639-8ed2-4c3d-9f85-7b83e88fab73 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:13.612675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-3915e639-8ed2-4c3d-9f85-7b83e88fab73 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:18:13.612675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:13.612675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:13.612675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:13.612675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:13.612675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:13.612675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:13.612675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:13.612675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:13.612675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:13.612675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:13.612675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:13.612675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:13.612675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:13.612675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:13.612675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:13.613632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:13.613632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:13.613632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:13.613632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:13.613632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:13.613632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:13.613632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:13.613632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:13.613632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:13.613632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:13.613632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:13.613632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:13.613632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:13.613632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:13.613632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:13.613632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:13.613632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:13.613632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:13.614664 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:13.614664 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:13.614664 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:13.614664 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:13.614664 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:13.614664 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:13.614664 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:13.614664 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:13.614664 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:18:13.614664 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:13.614664 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 271/555] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:18:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:13.622238 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915e639-8ed2-4c3d-9f85-7b83e88fab73 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:13.668422 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bcdb4b-6432-4b69-bbfa-016867ef8f5d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:13.718497 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:13.721328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61321791-7e8c-4122-9d8c-9415e5080391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:13.722057 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61321791-7e8c-4122-9d8c-9415e5080391 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:13.722446 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61321791-7e8c-4122-9d8c-9415e5080391 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:13.722784 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61321791-7e8c-4122-9d8c-9415e5080391 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:13.723508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61321791-7e8c-4122-9d8c-9415e5080391 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:13.723882 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61321791-7e8c-4122-9d8c-9415e5080391 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:13.725319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61321791-7e8c-4122-9d8c-9415e5080391 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:13.734771 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:13.739024 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 285/556] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:13.745319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61321791-7e8c-4122-9d8c-9415e5080391 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:13.801617 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915e639-8ed2-4c3d-9f85-7b83e88fab73 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:13.857933 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61321791-7e8c-4122-9d8c-9415e5080391 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:13.865719 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca9049-72ab-46c3-9c9e-5cd46d5e2e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:13.866440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca9049-72ab-46c3-9c9e-5cd46d5e2e9e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:13.866797 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca9049-72ab-46c3-9c9e-5cd46d5e2e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:13.867144 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca9049-72ab-46c3-9c9e-5cd46d5e2e9e None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:13.867986 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ca9049-72ab-46c3-9c9e-5cd46d5e2e9e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:13.868432 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ca9049-72ab-46c3-9c9e-5cd46d5e2e9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:13.869958 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ca9049-72ab-46c3-9c9e-5cd46d5e2e9e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:13.888669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 272/557] 10.222.0.58 () {56 vars in 1135 bytes} [Thu Sep 1 16:18:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:13.889603 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca9049-72ab-46c3-9c9e-5cd46d5e2e9e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:13.904730 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024b1ad-f407-412b-99cd-5dd95cc56047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:13.905360 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024b1ad-f407-412b-99cd-5dd95cc56047 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:13.905639 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024b1ad-f407-412b-99cd-5dd95cc56047 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:13.905909 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024b1ad-f407-412b-99cd-5dd95cc56047 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:13.916054 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024b1ad-f407-412b-99cd-5dd95cc56047 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:13.918383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024b1ad-f407-412b-99cd-5dd95cc56047 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:13.933633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-e024b1ad-f407-412b-99cd-5dd95cc56047 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:18:13.933633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:13.933633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:13.933633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:13.933633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:13.933633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:13.933633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:13.933633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:13.933633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:13.933633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:13.933633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:13.933633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:13.933633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:13.933633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:13.933633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:13.933633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:13.935171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:13.935171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:13.935171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:13.935171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:13.935171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:13.935171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:13.935171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:13.935171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:13.935171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:13.935171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:13.935171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:13.935171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:13.935171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:13.935171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:13.935171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:13.935171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:13.935171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:13.935171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:13.936114 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:13.936114 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:13.936114 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:13.936114 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:13.936114 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:13.936114 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:13.936114 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:13.936114 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:13.936114 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:18:13.936114 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:13.936114 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 286/558] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Sep 1 16:18:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:13.937156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024b1ad-f407-412b-99cd-5dd95cc56047 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:14.004789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:14.023125 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca9049-72ab-46c3-9c9e-5cd46d5e2e9e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:14.026184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:14.064329 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024b1ad-f407-412b-99cd-5dd95cc56047 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:14.077142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84052ada-7153-4910-92cc-d928a3dbe476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:14.078166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84052ada-7153-4910-92cc-d928a3dbe476 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:14.078681 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84052ada-7153-4910-92cc-d928a3dbe476 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:14.079098 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84052ada-7153-4910-92cc-d928a3dbe476 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:14.093157 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84052ada-7153-4910-92cc-d928a3dbe476 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:14.100403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-84052ada-7153-4910-92cc-d928a3dbe476 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:14.100403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:14.100403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:14.100403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:14.100403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:14.100403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:14.100403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:14.100403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:14.100403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:14.100403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:14.100403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:14.100403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:14.100403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:14.100403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:14.100403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:14.100403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:14.101526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:14.101526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:14.101526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:14.101526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:14.101526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:14.101526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:14.101526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:14.101526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:14.101526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:14.101526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:14.101526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:14.101526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:14.101526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:14.101526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:14.101526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:14.101526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:14.103427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:14.103427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:14.103427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:14.103427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:14.103427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:14.103427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:14.103427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:14.103427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:14.103427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:14.103427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:14.103427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:14.103427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:14.103427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 273/559] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Sep 1 16:18:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:14.103427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84052ada-7153-4910-92cc-d928a3dbe476 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:14.109253 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e0fd4-ad42-4c2b-bea0-2edf7aba5c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:14.110441 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e0fd4-ad42-4c2b-bea0-2edf7aba5c36 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:14.110883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e0fd4-ad42-4c2b-bea0-2edf7aba5c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:14.111303 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e0fd4-ad42-4c2b-bea0-2edf7aba5c36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:14.199295 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327e0fd4-ad42-4c2b-bea0-2edf7aba5c36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:14.201506 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327e0fd4-ad42-4c2b-bea0-2edf7aba5c36 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:14.231196 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84052ada-7153-4910-92cc-d928a3dbe476 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:14.275549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832448b7-dcde-4311-a814-21281c35670f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:14.276272 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832448b7-dcde-4311-a814-21281c35670f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:14.276557 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832448b7-dcde-4311-a814-21281c35670f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:14.276829 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832448b7-dcde-4311-a814-21281c35670f None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:14.277163 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:14.277859 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832448b7-dcde-4311-a814-21281c35670f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:14.277859 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832448b7-dcde-4311-a814-21281c35670f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:14.279942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832448b7-dcde-4311-a814-21281c35670f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:14.306423 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 274/560] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Sep 1 16:18:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:14.306423 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832448b7-dcde-4311-a814-21281c35670f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:14.331615 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:14.341784 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 287/561] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:18:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:18:14.342722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e0fd4-ad42-4c2b-bea0-2edf7aba5c36 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:14.414988 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832448b7-dcde-4311-a814-21281c35670f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:14.447743 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e0fd4-ad42-4c2b-bea0-2edf7aba5c36 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:14.449436 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea09f2-b101-449d-9215-38e920c03c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:14.449987 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea09f2-b101-449d-9215-38e920c03c12 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:14.450296 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea09f2-b101-449d-9215-38e920c03c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:14.450573 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea09f2-b101-449d-9215-38e920c03c12 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:14.451233 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea09f2-b101-449d-9215-38e920c03c12 None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:14.451539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea09f2-b101-449d-9215-38e920c03c12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:14.452898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea09f2-b101-449d-9215-38e920c03c12 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:14.460696 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 275/562] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Sep 1 16:18:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:14.461558 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea09f2-b101-449d-9215-38e920c03c12 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:14.485233 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29694f8e-f758-4140-bf73-7d357f656dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:14.485859 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29694f8e-f758-4140-bf73-7d357f656dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:14.486141 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29694f8e-f758-4140-bf73-7d357f656dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:14.486446 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29694f8e-f758-4140-bf73-7d357f656dd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:14.496913 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29694f8e-f758-4140-bf73-7d357f656dd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:14.498418 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29694f8e-f758-4140-bf73-7d357f656dd0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:14.513307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-29694f8e-f758-4140-bf73-7d357f656dd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:14.513307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:14.513307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:14.513307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:14.513307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:14.513307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:14.513307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:14.513307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:14.513307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:14.513307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:14.513307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:14.513307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:14.513307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:14.513307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:14.513307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:14.513307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:14.514305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:14.514305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:14.514305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:14.514305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:14.514305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:14.514305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:14.514305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:14.514305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:14.514305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:14.514305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:14.514305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:14.514305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:14.514305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:14.514305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:14.514305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:14.514305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:14.514305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:14.515268 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:14.515268 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:14.515268 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:14.515873 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 288/563] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:14.517258 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4429c-4aef-4244-bcdf-d31d4248e722 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:14.517548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4429c-4aef-4244-bcdf-d31d4248e722 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:14.518267 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4429c-4aef-4244-bcdf-d31d4248e722 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:14.519790 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 289/564] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:14.524751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac6662-e4c4-4a8c-b0e2-268fe68f3fe1 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:14.525046 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac6662-e4c4-4a8c-b0e2-268fe68f3fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:14.525324 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac6662-e4c4-4a8c-b0e2-268fe68f3fe1 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:14.526530 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 290/565] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:14.527215 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac6662-e4c4-4a8c-b0e2-268fe68f3fe1 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:14.542957 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:14.552241 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:14.568609 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea09f2-b101-449d-9215-38e920c03c12 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:14.600407 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67301375-ef37-4905-8d2d-8bb30b246935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:14.601023 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67301375-ef37-4905-8d2d-8bb30b246935 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:14.601307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67301375-ef37-4905-8d2d-8bb30b246935 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:14.601581 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67301375-ef37-4905-8d2d-8bb30b246935 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:14.612445 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67301375-ef37-4905-8d2d-8bb30b246935 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:14.613890 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67301375-ef37-4905-8d2d-8bb30b246935 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:14.628175 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-67301375-ef37-4905-8d2d-8bb30b246935 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:14.628175 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:14.628175 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:14.628175 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:14.628175 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:14.628175 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:14.628175 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:14.628175 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:14.628175 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:14.628175 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:14.628175 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:14.628175 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:14.628175 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:14.628175 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:14.628175 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:14.628175 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:14.629120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:14.629120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:14.629120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:14.629120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:14.629120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:14.629120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:14.629120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:14.629120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:14.629120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:14.629120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:14.629120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:14.629120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:14.629120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:14.629120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:14.629120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:14.629120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:14.629120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:14.630034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:14.630034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:14.630034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:14.630034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 276/566] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:14.630306 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac6662-e4c4-4a8c-b0e2-268fe68f3fe1 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:14.630385 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67301375-ef37-4905-8d2d-8bb30b246935 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:14.662479 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb46ee-50fd-4aea-8501-f5a0e202b2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:14.663098 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb46ee-50fd-4aea-8501-f5a0e202b2ac None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:14.663371 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb46ee-50fd-4aea-8501-f5a0e202b2ac None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:14.663648 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb46ee-50fd-4aea-8501-f5a0e202b2ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:14.674703 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb46ee-50fd-4aea-8501-f5a0e202b2ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:14.676165 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb46ee-50fd-4aea-8501-f5a0e202b2ac None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:14.690689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-f0bb46ee-50fd-4aea-8501-f5a0e202b2ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:14.690689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:14.690689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:14.690689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:14.690689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:14.690689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:14.690689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:14.690689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:14.690689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:14.690689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:14.690689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:14.690689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:14.690689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:14.690689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:14.690689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:14.690689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:14.691860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:14.691860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:14.691860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:14.691860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:14.691860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:14.691860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:14.691860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:14.691860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:14.691860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:14.691860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:14.691860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:14.691860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:14.691860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:14.691860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:14.691860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:14.691860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:14.691860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:14.692874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:14.692874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:14.692874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:14.692874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 291/567] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:14.693178 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494215b-430a-40d0-abab-7b790f888efb None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:14.693248 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494215b-430a-40d0-abab-7b790f888efb None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:14.693610 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494215b-430a-40d0-abab-7b790f888efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:14.711008 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b494215b-430a-40d0-abab-7b790f888efb None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:14.712174 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b494215b-430a-40d0-abab-7b790f888efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:14.733660 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67301375-ef37-4905-8d2d-8bb30b246935 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:14.766936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc15dc9-219f-45e0-a717-37d517d7b206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:14.767537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc15dc9-219f-45e0-a717-37d517d7b206 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:14.767818 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc15dc9-219f-45e0-a717-37d517d7b206 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:14.768091 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc15dc9-219f-45e0-a717-37d517d7b206 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:14.768733 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc15dc9-219f-45e0-a717-37d517d7b206 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:14.769037 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc15dc9-219f-45e0-a717-37d517d7b206 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:14.770430 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc15dc9-219f-45e0-a717-37d517d7b206 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:14.770653 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:14.772731 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:14.780279 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 292/568] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:18:14.781478 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43ab86-e559-4b38-aa67-73a5f7aa7720 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:14.781759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43ab86-e559-4b38-aa67-73a5f7aa7720 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:14.782031 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43ab86-e559-4b38-aa67-73a5f7aa7720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:14.787760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 277/569] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:14.787760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc15dc9-219f-45e0-a717-37d517d7b206 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:14.804713 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-9e43ab86-e559-4b38-aa67-73a5f7aa7720 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:14.806530 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9e43ab86-e559-4b38-aa67-73a5f7aa7720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:14.878846 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 293/570] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:18:14.882775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43ab86-e559-4b38-aa67-73a5f7aa7720 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:14.891955 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc15dc9-219f-45e0-a717-37d517d7b206 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:14.931543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f626b-f84b-4979-9f2d-3ca9f017c47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:14.932200 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f626b-f84b-4979-9f2d-3ca9f017c47b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:14.932489 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f626b-f84b-4979-9f2d-3ca9f017c47b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:14.932813 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f626b-f84b-4979-9f2d-3ca9f017c47b None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:14.933468 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117f626b-f84b-4979-9f2d-3ca9f017c47b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:14.933771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117f626b-f84b-4979-9f2d-3ca9f017c47b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:14.935389 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117f626b-f84b-4979-9f2d-3ca9f017c47b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:14.952453 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 278/571] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:14.952453 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f626b-f84b-4979-9f2d-3ca9f017c47b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:15.008962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:15.028018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43ab86-e559-4b38-aa67-73a5f7aa7720 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:15.059082 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f626b-f84b-4979-9f2d-3ca9f017c47b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:15.059568 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:15.077396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19f1f6-baa9-4265-aa43-ad63aae8594a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:15.078014 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19f1f6-baa9-4265-aa43-ad63aae8594a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:15.078326 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19f1f6-baa9-4265-aa43-ad63aae8594a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:15.078605 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19f1f6-baa9-4265-aa43-ad63aae8594a None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:15.079257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19f1f6-baa9-4265-aa43-ad63aae8594a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:15.079564 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19f1f6-baa9-4265-aa43-ad63aae8594a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:15.081432 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19f1f6-baa9-4265-aa43-ad63aae8594a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:15.097279 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 294/572] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:15.097279 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19f1f6-baa9-4265-aa43-ad63aae8594a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:15.098439 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3599529-da58-456f-bef3-7aa48b200e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:15.099076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3599529-da58-456f-bef3-7aa48b200e20 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:15.099357 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3599529-da58-456f-bef3-7aa48b200e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:15.099630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3599529-da58-456f-bef3-7aa48b200e20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:15.106652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3599529-da58-456f-bef3-7aa48b200e20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:15.108082 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3599529-da58-456f-bef3-7aa48b200e20 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:15.120318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-d3599529-da58-456f-bef3-7aa48b200e20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:15.120318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:15.120318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:15.120318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:15.120318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:15.120318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:15.120318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:15.120318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:15.120318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:15.120318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:15.120318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:15.120318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:15.120318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:15.120318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:15.120318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:15.121224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:15.121224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:15.121224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:15.121224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:15.121224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:15.121224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:15.121224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:15.121224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:15.121224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:15.121224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:15.121224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:15.121224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:15.121224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:15.121224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:15.121224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:15.121224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:15.121224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:15.122154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:15.122154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:15.122154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:15.122154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:15.122154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:15.122154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:15.122154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:15.122154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:15.122154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 279/573] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:15.122690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3599529-da58-456f-bef3-7aa48b200e20 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:15.207720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19f1f6-baa9-4265-aa43-ad63aae8594a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:15.224425 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3599529-da58-456f-bef3-7aa48b200e20 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:15.239861 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:15.240016 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ee9b2-3df8-47df-afb1-8b5e13e21612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:15.240650 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ee9b2-3df8-47df-afb1-8b5e13e21612 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:15.241009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ee9b2-3df8-47df-afb1-8b5e13e21612 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:15.241253 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ee9b2-3df8-47df-afb1-8b5e13e21612 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:15.248561 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ee9b2-3df8-47df-afb1-8b5e13e21612 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:15.250019 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ee9b2-3df8-47df-afb1-8b5e13e21612 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:15.257865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db2ead-73a2-43c9-ae6b-26c02e214d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:15.259182 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db2ead-73a2-43c9-ae6b-26c02e214d54 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:15.259411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db2ead-73a2-43c9-ae6b-26c02e214d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:15.259685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db2ead-73a2-43c9-ae6b-26c02e214d54 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:15.261709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-551ee9b2-3df8-47df-afb1-8b5e13e21612 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:15.261709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:15.261709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:15.261709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:15.261709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:15.261709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:15.261709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:15.261709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:15.261709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:15.261709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:15.261709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:15.261709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:15.261709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:15.261709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:15.261709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:15.262654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:15.262654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:15.262654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:15.262654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:15.262654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:15.262654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:15.262654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:15.262654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:15.262654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:15.262654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:15.262654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:15.262654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:15.262654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:15.262654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:15.262654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:15.262654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:15.262654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:15.263564 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:15.263564 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:15.263564 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:15.263564 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:15.263564 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:15.263564 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:15.263564 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:15.263564 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:15.263564 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 295/574] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:15.263564 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ee9b2-3df8-47df-afb1-8b5e13e21612 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:15.273317 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db2ead-73a2-43c9-ae6b-26c02e214d54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:15.279368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-29db2ead-73a2-43c9-ae6b-26c02e214d54 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:15.279368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:15.279368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:15.279368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:15.279368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:15.279368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:15.279368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:15.279368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:15.279368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:15.279368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:15.279368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:15.279368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:15.279368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:15.279368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:15.279368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:15.279368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:15.280494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:15.280494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:15.280494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:15.280494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:15.280494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:15.280494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:15.280494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:15.280494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:15.280494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:15.280494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:15.280494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:15.280494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:15.280494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:15.280494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:15.280494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:15.280494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:15.281341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:15.281341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:15.281341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:15.281341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:15.281341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:15.281341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:15.281341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:15.281341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:15.281341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:15.281341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:15.281341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:15.281341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:15.281341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 280/575] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Sep 1 16:18:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:15.282040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db2ead-73a2-43c9-ae6b-26c02e214d54 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:15.313802 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:15.371981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ee9b2-3df8-47df-afb1-8b5e13e21612 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:15.381252 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db2ead-73a2-43c9-ae6b-26c02e214d54 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:15.404831 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9555608e-2648-4075-ac41-943b0a09de9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:15.405459 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9555608e-2648-4075-ac41-943b0a09de9f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:15.405737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9555608e-2648-4075-ac41-943b0a09de9f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:15.406005 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9555608e-2648-4075-ac41-943b0a09de9f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:15.413257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9555608e-2648-4075-ac41-943b0a09de9f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:15.413724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48163b-a09a-4ca0-b073-aa342230bb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:15.414839 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48163b-a09a-4ca0-b073-aa342230bb5e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:15.415117 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48163b-a09a-4ca0-b073-aa342230bb5e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:15.415642 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9555608e-2648-4075-ac41-943b0a09de9f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:15.415750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48163b-a09a-4ca0-b073-aa342230bb5e None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:15.416063 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48163b-a09a-4ca0-b073-aa342230bb5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:15.416365 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48163b-a09a-4ca0-b073-aa342230bb5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:15.418043 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48163b-a09a-4ca0-b073-aa342230bb5e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:15.426876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-9555608e-2648-4075-ac41-943b0a09de9f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 16:18:15.426876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:15.426876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:15.426876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:15.426876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:15.426876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:15.426876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:15.426876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:15.426876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:15.426876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:15.426876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:15.426876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:15.426876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:15.426876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:15.426876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:15.426876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:15.427983 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:15.427983 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:15.427983 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:15.427983 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:15.427983 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:15.427983 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:15.427983 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:15.427983 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:15.427983 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:15.427983 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:15.427983 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:15.427983 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:15.427983 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:15.427983 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:15.427983 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:15.427983 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:15.427983 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:15.427983 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:15.429194 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:15.429194 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:15.429194 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:15.429194 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:15.429194 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:15.429194 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:15.429194 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:15.429194 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:15.429194 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 16:18:15.429194 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:15.429194 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 296/576] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:18:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:15.429194 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9555608e-2648-4075-ac41-943b0a09de9f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:15.430841 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 281/577] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:15.431572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48163b-a09a-4ca0-b073-aa342230bb5e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:15.465815 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:15.531330 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48163b-a09a-4ca0-b073-aa342230bb5e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:15.531907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9555608e-2648-4075-ac41-943b0a09de9f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:15.534182 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:15.566431 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d80d5-0428-4553-80bb-438ccb09cd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:15.566609 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e242a9-0a48-4d23-93a7-ac330ae9be3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:15.567028 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d80d5-0428-4553-80bb-438ccb09cd5a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:15.567127 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e242a9-0a48-4d23-93a7-ac330ae9be3e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:15.567310 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d80d5-0428-4553-80bb-438ccb09cd5a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:15.567399 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e242a9-0a48-4d23-93a7-ac330ae9be3e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:15.567584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d80d5-0428-4553-80bb-438ccb09cd5a None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:15.567684 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e242a9-0a48-4d23-93a7-ac330ae9be3e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:15.568274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d80d5-0428-4553-80bb-438ccb09cd5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:15.568581 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d80d5-0428-4553-80bb-438ccb09cd5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:15.570019 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d80d5-0428-4553-80bb-438ccb09cd5a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:15.577598 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e242a9-0a48-4d23-93a7-ac330ae9be3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:15.581531 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 282/578] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:15.582332 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d80d5-0428-4553-80bb-438ccb09cd5a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:15.582900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-02e242a9-0a48-4d23-93a7-ac330ae9be3e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:15.582900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:15.582900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:15.582900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:15.582900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:15.582900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:15.582900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:15.582900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:15.582900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:15.582900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:15.582900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:15.582900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:15.582900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:15.582900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:15.582900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:15.582900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:15.583834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:15.583834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:15.583834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:15.583834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:15.583834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:15.583834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:15.583834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:15.583834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:15.583834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:15.583834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:15.583834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:15.583834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:15.583834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:15.583834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:15.583834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:15.583834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:15.585333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:15.585333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:15.585333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:15.585333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:15.585333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:15.585333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:15.585333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:15.585333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:15.585333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:15.585333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:15.585333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:15.585333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:15.585333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 297/579] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Sep 1 16:18:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:15.585333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e242a9-0a48-4d23-93a7-ac330ae9be3e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:15.678916 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:15.685740 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d80d5-0428-4553-80bb-438ccb09cd5a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:15.686212 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e242a9-0a48-4d23-93a7-ac330ae9be3e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:15.718174 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624e113-5cdc-4df3-b43e-e76e51a0fade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:15.718504 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d5862-e018-4f90-bf79-c9b1bf386be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:15.718794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624e113-5cdc-4df3-b43e-e76e51a0fade None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:15.719076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624e113-5cdc-4df3-b43e-e76e51a0fade None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:15.719181 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d5862-e018-4f90-bf79-c9b1bf386be2 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:15.719355 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624e113-5cdc-4df3-b43e-e76e51a0fade None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:15.719499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d5862-e018-4f90-bf79-c9b1bf386be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:15.719658 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d5862-e018-4f90-bf79-c9b1bf386be2 None admin] PATH_INFO: `/v3/projects/479530ced580426b85d1fc07af43417f/users/1f0c99fdf8bd4331ad7e641ee38554c1/roles/b0923c7a240149ae9ece5b3c1622e860` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:15.720017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624e113-5cdc-4df3-b43e-e76e51a0fade None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:15.720324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624e113-5cdc-4df3-b43e-e76e51a0fade None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:15.721680 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624e113-5cdc-4df3-b43e-e76e51a0fade None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:15.729334 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 283/580] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Sep 1 16:18:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:15.730108 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624e113-5cdc-4df3-b43e-e76e51a0fade None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:15.740403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d5862-e018-4f90-bf79-c9b1bf386be2 None admin] RBAC: Authorizing `identity:create_grant(project_id=479530ced580426b85d1fc07af43417f, user_id=1f0c99fdf8bd4331ad7e641ee38554c1, role_id=b0923c7a240149ae9ece5b3c1622e860)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:15.741848 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d5862-e018-4f90-bf79-c9b1bf386be2 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:15.754881 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 298/581] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Sep 1 16:18:15 2022] PUT /identity/v3/projects/479530ced580426b85d1fc07af43417f/users/1f0c99fdf8bd4331ad7e641ee38554c1/roles/b0923c7a240149ae9ece5b3c1622e860 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:18:15.755769 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d5862-e018-4f90-bf79-c9b1bf386be2 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:15.785798 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:15.846486 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624e113-5cdc-4df3-b43e-e76e51a0fade None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:15.886012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d5862-e018-4f90-bf79-c9b1bf386be2 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:15.886461 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e1730-400b-4c87-919b-4e1ebcb79ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:15.887087 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e1730-400b-4c87-919b-4e1ebcb79ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:15.887372 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e1730-400b-4c87-919b-4e1ebcb79ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:15.887648 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e1730-400b-4c87-919b-4e1ebcb79ba2 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:15.888398 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e1730-400b-4c87-919b-4e1ebcb79ba2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:15.888708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e1730-400b-4c87-919b-4e1ebcb79ba2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:15.890177 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e1730-400b-4c87-919b-4e1ebcb79ba2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:15.916194 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 284/582] 10.222.0.58 () {56 vars in 1137 bytes} [Thu Sep 1 16:18:15 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:15.917313 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e1730-400b-4c87-919b-4e1ebcb79ba2 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:15.930634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc2e754-83b3-4faf-beb3-53a33f46be3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:15.931246 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2e754-83b3-4faf-beb3-53a33f46be3b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:15.931532 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2e754-83b3-4faf-beb3-53a33f46be3b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:15.931809 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2e754-83b3-4faf-beb3-53a33f46be3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:15.932569 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc2e754-83b3-4faf-beb3-53a33f46be3b None admin] RBAC: Adding query filter params (role.id=b0923c7a240149ae9ece5b3c1622e860, scope.project.id=479530ced580426b85d1fc07af43417f, user.id=e789695517c1437fb224f5a6195655f9) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:15.932892 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc2e754-83b3-4faf-beb3-53a33f46be3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:15.934347 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc2e754-83b3-4faf-beb3-53a33f46be3b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:15.935857 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:15.942688 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 299/583] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:18:15 2022] GET /identity/v3/role_assignments?user.id=e789695517c1437fb224f5a6195655f9&scope.project.id=479530ced580426b85d1fc07af43417f&role.id=b0923c7a240149ae9ece5b3c1622e860 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:15.942688 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc2e754-83b3-4faf-beb3-53a33f46be3b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:16.036512 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e1730-400b-4c87-919b-4e1ebcb79ba2 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:16.065590 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:16.090852 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c9d0ac-3064-496e-9b3d-f7e8ab8c802e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:16.091612 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9d0ac-3064-496e-9b3d-f7e8ab8c802e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:16.091901 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9d0ac-3064-496e-9b3d-f7e8ab8c802e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:16.092013 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc2e754-83b3-4faf-beb3-53a33f46be3b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:16.092178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9d0ac-3064-496e-9b3d-f7e8ab8c802e None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:16.092860 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c9d0ac-3064-496e-9b3d-f7e8ab8c802e None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:16.093254 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c9d0ac-3064-496e-9b3d-f7e8ab8c802e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:16.095830 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c9d0ac-3064-496e-9b3d-f7e8ab8c802e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:16.112942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 285/584] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Sep 1 16:18:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:16.114727 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c9d0ac-3064-496e-9b3d-f7e8ab8c802e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:16.132620 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fdf5e9-5c92-4b52-85f4-e525a16c667f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:16.133858 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdf5e9-5c92-4b52-85f4-e525a16c667f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:16.134140 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdf5e9-5c92-4b52-85f4-e525a16c667f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:16.134438 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdf5e9-5c92-4b52-85f4-e525a16c667f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:16.148064 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fdf5e9-5c92-4b52-85f4-e525a16c667f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:16.150716 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fdf5e9-5c92-4b52-85f4-e525a16c667f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:16.167555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-26fdf5e9-5c92-4b52-85f4-e525a16c667f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:16.167555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:16.167555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:16.167555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:16.167555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:16.167555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:16.167555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:16.167555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:16.167555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:16.167555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:16.167555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:16.167555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:16.167555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:16.167555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:16.167555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:16.167555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:16.168562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:16.168562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:16.168562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:16.168562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:16.168562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:16.168562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:16.168562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:16.168562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:16.168562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:16.168562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:16.168562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:16.168562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:16.168562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:16.168562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:16.168562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:16.168562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:16.168562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:16.169482 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:16.169482 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:16.169482 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:16.169482 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 300/585] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:16.169482 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fdf5e9-5c92-4b52-85f4-e525a16c667f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:16.229645 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:16.260960 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c9d0ac-3064-496e-9b3d-f7e8ab8c802e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:16.287949 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fdf5e9-5c92-4b52-85f4-e525a16c667f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:16.296891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d946b5-640f-4a3a-8dae-7680af564fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:16.297973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d946b5-640f-4a3a-8dae-7680af564fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:16.297973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d946b5-640f-4a3a-8dae-7680af564fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:16.298416 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d946b5-640f-4a3a-8dae-7680af564fe8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:16.309524 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d946b5-640f-4a3a-8dae-7680af564fe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:16.311145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d946b5-640f-4a3a-8dae-7680af564fe8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:16.323893 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:16.326842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-a4d946b5-640f-4a3a-8dae-7680af564fe8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:16.326842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:16.326842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:16.326842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:16.326842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:16.326842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:16.326842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:16.326842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:16.326842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:16.326842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:16.326842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:16.326842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:16.326842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:16.326842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:16.326842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:16.326842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:16.327781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:16.327781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:16.327781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:16.327781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:16.327781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:16.327781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:16.327781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:16.327781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:16.327781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:16.327781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:16.327781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:16.327781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:16.327781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:16.327781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:16.327781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:16.327781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:16.327781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:16.328698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:16.328698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:16.328698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:16.332238 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 286/586] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:16.333420 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d946b5-640f-4a3a-8dae-7680af564fe8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:16.333716 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb2981-b03e-44e3-989d-88d410cc4a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:16.335044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb2981-b03e-44e3-989d-88d410cc4a66 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:16.335524 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb2981-b03e-44e3-989d-88d410cc4a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:16.335991 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb2981-b03e-44e3-989d-88d410cc4a66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:16.350249 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb2981-b03e-44e3-989d-88d410cc4a66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:16.352977 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb2981-b03e-44e3-989d-88d410cc4a66 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:16.372059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-d9fb2981-b03e-44e3-989d-88d410cc4a66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:16.372059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:16.372059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:16.372059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:16.372059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:16.372059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:16.372059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:16.372059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:16.372059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:16.372059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:16.372059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:16.372059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:16.372059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:16.372059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:16.372059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:16.372059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:16.373058 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:16.373058 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:16.373058 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:16.373058 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:16.373058 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:16.373058 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:16.373058 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:16.373058 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:16.373058 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:16.373058 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:16.373058 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:16.373058 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:16.373058 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:16.373058 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:16.373058 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:16.373058 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:16.373058 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:16.373969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:16.373969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:16.373969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:16.380221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 301/587] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:16.380221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb2981-b03e-44e3-989d-88d410cc4a66 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:16.478378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d946b5-640f-4a3a-8dae-7680af564fe8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:16.505944 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb2981-b03e-44e3-989d-88d410cc4a66 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:16.512696 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:16.519209 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09fa45f-0dac-4a90-8e6b-3a967dd9cd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:16.520106 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fa45f-0dac-4a90-8e6b-3a967dd9cd10 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:16.520620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fa45f-0dac-4a90-8e6b-3a967dd9cd10 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:16.521094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fa45f-0dac-4a90-8e6b-3a967dd9cd10 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:16.521936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09fa45f-0dac-4a90-8e6b-3a967dd9cd10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:16.522485 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09fa45f-0dac-4a90-8e6b-3a967dd9cd10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:16.524168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09fa45f-0dac-4a90-8e6b-3a967dd9cd10 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:16.536694 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 287/588] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:16.537845 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09fa45f-0dac-4a90-8e6b-3a967dd9cd10 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:16.549929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6eecb-b17a-4ed6-88fe-50ac1ef4db5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:16.550506 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6eecb-b17a-4ed6-88fe-50ac1ef4db5b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:16.550797 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6eecb-b17a-4ed6-88fe-50ac1ef4db5b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:16.551055 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6eecb-b17a-4ed6-88fe-50ac1ef4db5b None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:16.551807 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6eecb-b17a-4ed6-88fe-50ac1ef4db5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:16.552114 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6eecb-b17a-4ed6-88fe-50ac1ef4db5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:16.553525 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6eecb-b17a-4ed6-88fe-50ac1ef4db5b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:16.567466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 302/589] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:16.568106 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6eecb-b17a-4ed6-88fe-50ac1ef4db5b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:16.615245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:16.643378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09fa45f-0dac-4a90-8e6b-3a967dd9cd10 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:16.678046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37189716-e835-418a-907f-d69bdf4764d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:16.678534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6eecb-b17a-4ed6-88fe-50ac1ef4db5b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:16.679257 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37189716-e835-418a-907f-d69bdf4764d9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:16.679733 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37189716-e835-418a-907f-d69bdf4764d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:16.680707 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37189716-e835-418a-907f-d69bdf4764d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:16.681536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37189716-e835-418a-907f-d69bdf4764d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:16.682044 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37189716-e835-418a-907f-d69bdf4764d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:16.683617 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37189716-e835-418a-907f-d69bdf4764d9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:16.700099 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 288/590] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:16.701185 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37189716-e835-418a-907f-d69bdf4764d9 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:16.715923 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68aae40-b417-4d11-a093-124b30be9a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:16.716555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68aae40-b417-4d11-a093-124b30be9a0e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:16.716975 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68aae40-b417-4d11-a093-124b30be9a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:16.717255 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68aae40-b417-4d11-a093-124b30be9a0e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:16.725230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68aae40-b417-4d11-a093-124b30be9a0e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:16.726796 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68aae40-b417-4d11-a093-124b30be9a0e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:16.739552 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-b68aae40-b417-4d11-a093-124b30be9a0e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:18:16.739552 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:16.739552 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:16.739552 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:16.739552 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:16.739552 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:16.739552 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:16.739552 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:16.739552 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:16.739552 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:16.739552 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:16.739552 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:16.739552 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:16.739552 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:16.739552 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:16.739552 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:16.740513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:16.740513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:16.740513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:16.740513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:16.740513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:16.740513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:16.740513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:16.740513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:16.740513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:16.740513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:16.740513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:16.740513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:16.740513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:16.740513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:16.740513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:16.740513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:16.740513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:16.740513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:16.742072 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:16.742072 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:16.742072 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:16.742072 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:16.742072 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:16.742072 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:16.742072 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:16.742072 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:16.742072 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:18:16.742072 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:16.745871 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 303/591] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:18:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:16.746755 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68aae40-b417-4d11-a093-124b30be9a0e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:16.759783 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:16.811839 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37189716-e835-418a-907f-d69bdf4764d9 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:16.853134 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9baa0-8700-47c3-aedf-e4fa15abd422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:16.854121 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9baa0-8700-47c3-aedf-e4fa15abd422 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:16.854629 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9baa0-8700-47c3-aedf-e4fa15abd422 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:16.855097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9baa0-8700-47c3-aedf-e4fa15abd422 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:16.862222 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd9baa0-8700-47c3-aedf-e4fa15abd422 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:16.862659 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68aae40-b417-4d11-a093-124b30be9a0e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:16.864144 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd9baa0-8700-47c3-aedf-e4fa15abd422 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:16.864383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:16.876966 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-9fd9baa0-8700-47c3-aedf-e4fa15abd422 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:16.876966 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:16.876966 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:16.876966 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:16.876966 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:16.876966 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:16.876966 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:16.876966 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:16.876966 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:16.876966 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:16.876966 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:16.876966 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:16.876966 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:16.876966 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:16.876966 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:16.877906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:16.877906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:16.877906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:16.877906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:16.877906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:16.877906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:16.877906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:16.877906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:16.877906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:16.877906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:16.877906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:16.877906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:16.877906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:16.877906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:16.877906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:16.877906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:16.877906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:16.878831 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:16.878831 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:16.878831 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:16.878831 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:16.878831 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:16.878831 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:16.878831 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:16.878831 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:16.881186 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 289/592] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:16.882306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9baa0-8700-47c3-aedf-e4fa15abd422 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:16.903555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3163b-835d-4fdc-9d7f-c643b3689bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:16.904198 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3163b-835d-4fdc-9d7f-c643b3689bc9 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:16.904482 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3163b-835d-4fdc-9d7f-c643b3689bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:16.904758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3163b-835d-4fdc-9d7f-c643b3689bc9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:16.915050 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a3163b-835d-4fdc-9d7f-c643b3689bc9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:16.916608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a3163b-835d-4fdc-9d7f-c643b3689bc9 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:16.929112 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-86a3163b-835d-4fdc-9d7f-c643b3689bc9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 16:18:16.929112 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:16.929112 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:16.929112 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:16.929112 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:16.929112 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:16.929112 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:16.929112 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:16.929112 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:16.929112 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:16.929112 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:16.929112 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:16.929112 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:16.929112 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:16.929112 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:16.929112 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:16.930891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:16.930891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:16.930891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:16.930891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:16.930891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:16.930891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:16.930891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:16.930891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:16.930891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:16.930891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:16.930891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:16.930891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:16.930891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:16.930891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:16.930891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:16.930891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:16.930891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:16.930891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:16.931929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:16.931929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:16.931929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:16.931929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:16.931929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:16.931929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:16.931929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:16.931929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:16.931929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 16:18:16.931929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:16.935933 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 304/593] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Sep 1 16:18:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:16.937292 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3163b-835d-4fdc-9d7f-c643b3689bc9 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:16.995190 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9baa0-8700-47c3-aedf-e4fa15abd422 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:17.005922 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:17.029937 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97e0b6-3f9b-4e90-873f-75d287858fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:17.030571 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97e0b6-3f9b-4e90-873f-75d287858fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:17.030848 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97e0b6-3f9b-4e90-873f-75d287858fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:17.031120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97e0b6-3f9b-4e90-873f-75d287858fb5 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:17.031847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97e0b6-3f9b-4e90-873f-75d287858fb5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:17.032159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97e0b6-3f9b-4e90-873f-75d287858fb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:17.033583 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97e0b6-3f9b-4e90-873f-75d287858fb5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:17.057207 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 290/594] 10.222.0.58 () {56 vars in 1137 bytes} [Thu Sep 1 16:18:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:17.057207 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97e0b6-3f9b-4e90-873f-75d287858fb5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:17.066299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3163b-835d-4fdc-9d7f-c643b3689bc9 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:17.104487 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7cfac8-24c0-4724-a5bf-604bc7126562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:17.105317 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7cfac8-24c0-4724-a5bf-604bc7126562 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:17.105842 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7cfac8-24c0-4724-a5bf-604bc7126562 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:17.106321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7cfac8-24c0-4724-a5bf-604bc7126562 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:17.107257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7cfac8-24c0-4724-a5bf-604bc7126562 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:17.107712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7cfac8-24c0-4724-a5bf-604bc7126562 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:17.109306 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7cfac8-24c0-4724-a5bf-604bc7126562 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:17.123649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 305/595] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:17.124873 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7cfac8-24c0-4724-a5bf-604bc7126562 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:17.151495 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:17.157700 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97e0b6-3f9b-4e90-873f-75d287858fb5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:17.193550 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97670703-7bf5-4c21-afb2-0ffbca13b838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:17.194114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97670703-7bf5-4c21-afb2-0ffbca13b838 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:17.194427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97670703-7bf5-4c21-afb2-0ffbca13b838 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:17.194855 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97670703-7bf5-4c21-afb2-0ffbca13b838 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:17.196134 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97670703-7bf5-4c21-afb2-0ffbca13b838 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:17.196442 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97670703-7bf5-4c21-afb2-0ffbca13b838 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:17.197881 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97670703-7bf5-4c21-afb2-0ffbca13b838 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:17.218625 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 291/596] 10.222.0.58 () {56 vars in 1139 bytes} [Thu Sep 1 16:18:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:17.219143 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97670703-7bf5-4c21-afb2-0ffbca13b838 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:17.241559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7cfac8-24c0-4724-a5bf-604bc7126562 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:17.255490 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:17.279418 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f3133-70c0-4b9a-8126-b4a4a21b8273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:17.280173 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f3133-70c0-4b9a-8126-b4a4a21b8273 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:17.280550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f3133-70c0-4b9a-8126-b4a4a21b8273 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:17.280918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f3133-70c0-4b9a-8126-b4a4a21b8273 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:17.293880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f3133-70c0-4b9a-8126-b4a4a21b8273 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:17.295771 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f3133-70c0-4b9a-8126-b4a4a21b8273 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:17.315458 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-3a7f3133-70c0-4b9a-8126-b4a4a21b8273 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:17.315458 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:17.315458 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:17.315458 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:17.315458 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:17.315458 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:17.315458 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:17.315458 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:17.315458 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:17.315458 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:17.315458 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:17.315458 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:17.315458 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:17.315458 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:17.315458 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:17.315458 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:17.316663 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:17.316663 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:17.316663 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:17.316663 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:17.316663 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:17.316663 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:17.316663 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:17.316663 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:17.316663 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:17.316663 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:17.316663 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:17.316663 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:17.316663 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:17.316663 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:17.316663 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:17.316663 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:17.316663 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:17.317672 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:17.317672 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:17.317672 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:17.322290 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 306/597] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:17.323279 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f3133-70c0-4b9a-8126-b4a4a21b8273 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:17.349217 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97670703-7bf5-4c21-afb2-0ffbca13b838 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:17.393237 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9434901-ad42-40be-8148-09e64dd875c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:17.394492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9434901-ad42-40be-8148-09e64dd875c7 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:17.395198 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9434901-ad42-40be-8148-09e64dd875c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:17.395663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9434901-ad42-40be-8148-09e64dd875c7 None admin] PATH_INFO: `/v3/projects/479530ced580426b85d1fc07af43417f/users/6a2c872129414105a112661cc2b477e6/roles/b0923c7a240149ae9ece5b3c1622e860` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:17.414005 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9434901-ad42-40be-8148-09e64dd875c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=479530ced580426b85d1fc07af43417f, user_id=6a2c872129414105a112661cc2b477e6, role_id=b0923c7a240149ae9ece5b3c1622e860)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:17.415993 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9434901-ad42-40be-8148-09e64dd875c7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:17.425492 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:17.432620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 292/598] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Sep 1 16:18:17 2022] PUT /identity/v3/projects/479530ced580426b85d1fc07af43417f/users/6a2c872129414105a112661cc2b477e6/roles/b0923c7a240149ae9ece5b3c1622e860 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:18:17.433831 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9434901-ad42-40be-8148-09e64dd875c7 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:17.458078 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f3133-70c0-4b9a-8126-b4a4a21b8273 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:17.499537 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7756a0f-f295-4e0a-aedc-fb65b4f09c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:17.500562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7756a0f-f295-4e0a-aedc-fb65b4f09c37 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:17.501071 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7756a0f-f295-4e0a-aedc-fb65b4f09c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:17.501529 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7756a0f-f295-4e0a-aedc-fb65b4f09c37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:17.514680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7756a0f-f295-4e0a-aedc-fb65b4f09c37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:17.516623 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7756a0f-f295-4e0a-aedc-fb65b4f09c37 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:17.524414 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:17.542720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-e7756a0f-f295-4e0a-aedc-fb65b4f09c37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:17.542720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:17.542720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:17.542720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:17.542720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:17.542720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:17.542720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:17.542720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:17.542720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:17.542720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:17.542720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:17.542720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:17.542720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:17.542720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:17.542720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:17.542720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:17.543758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:17.543758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:17.543758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:17.543758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:17.543758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:17.543758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:17.543758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:17.543758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:17.543758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:17.543758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:17.543758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:17.543758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:17.543758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:17.543758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:17.543758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:17.543758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:17.543758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:17.544665 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:17.544665 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:17.544665 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:17.546549 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 307/599] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:17.552515 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7756a0f-f295-4e0a-aedc-fb65b4f09c37 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:17.560977 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9434901-ad42-40be-8148-09e64dd875c7 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:17.608113 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389cae7b-945f-4f7c-9384-c9667bfc6cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:17.608665 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cae7b-945f-4f7c-9384-c9667bfc6cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:17.608949 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cae7b-945f-4f7c-9384-c9667bfc6cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:17.609225 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cae7b-945f-4f7c-9384-c9667bfc6cb9 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:17.609972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389cae7b-945f-4f7c-9384-c9667bfc6cb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:17.610216 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389cae7b-945f-4f7c-9384-c9667bfc6cb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:17.611772 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389cae7b-945f-4f7c-9384-c9667bfc6cb9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:17.631360 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 293/600] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:17.637575 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389cae7b-945f-4f7c-9384-c9667bfc6cb9 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:17.687656 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7756a0f-f295-4e0a-aedc-fb65b4f09c37 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:17.722376 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:17.735120 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5ce17-5a19-480f-9694-0f795b56ad84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:17.736341 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5ce17-5a19-480f-9694-0f795b56ad84 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:17.736906 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5ce17-5a19-480f-9694-0f795b56ad84 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:17.737595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5ce17-5a19-480f-9694-0f795b56ad84 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:17.738478 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5ce17-5a19-480f-9694-0f795b56ad84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:17.738590 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5ce17-5a19-480f-9694-0f795b56ad84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:17.739958 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5ce17-5a19-480f-9694-0f795b56ad84 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:17.756507 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 308/601] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:18:17.763965 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5ce17-5a19-480f-9694-0f795b56ad84 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:17.770433 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389cae7b-945f-4f7c-9384-c9667bfc6cb9 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:17.807106 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e521d-be06-4759-86ae-90a06b63b55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:17.807741 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e521d-be06-4759-86ae-90a06b63b55c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:17.808086 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e521d-be06-4759-86ae-90a06b63b55c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:17.808369 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e521d-be06-4759-86ae-90a06b63b55c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:17.811156 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:17.816735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e521d-be06-4759-86ae-90a06b63b55c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:17.819366 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e521d-be06-4759-86ae-90a06b63b55c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:17.832354 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-3e1e521d-be06-4759-86ae-90a06b63b55c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:17.832354 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:17.832354 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:17.832354 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:17.832354 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:17.832354 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:17.832354 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:17.832354 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:17.832354 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:17.832354 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:17.832354 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:17.832354 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:17.832354 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:17.832354 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:17.832354 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:17.833513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:17.833513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:17.833513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:17.833513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:17.833513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:17.833513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:17.833513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:17.833513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:17.833513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:17.833513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:17.833513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:17.833513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:17.833513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:17.833513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:17.833513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:17.833513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:17.833513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:17.834686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:17.834686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:17.834686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:17.834686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:17.834686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:17.834686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:17.834686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:17.834686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:17.841704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 294/602] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:17.843225 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a89c1a-1935-453f-8f30-40a4b06d7c6e None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:17.843667 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a89c1a-1935-453f-8f30-40a4b06d7c6e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:17.844087 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a89c1a-1935-453f-8f30-40a4b06d7c6e None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:17.846280 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 295/603] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:17.847279 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a89c1a-1935-453f-8f30-40a4b06d7c6e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:17.893029 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5ce17-5a19-480f-9694-0f795b56ad84 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:17.937530 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126cd004-686d-4679-aca7-8d8d1b930029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:17.938622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cd004-686d-4679-aca7-8d8d1b930029 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:17.938913 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cd004-686d-4679-aca7-8d8d1b930029 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:17.939282 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cd004-686d-4679-aca7-8d8d1b930029 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:17.955435 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126cd004-686d-4679-aca7-8d8d1b930029 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:17.957999 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126cd004-686d-4679-aca7-8d8d1b930029 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:17.960693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a89c1a-1935-453f-8f30-40a4b06d7c6e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:17.972960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-126cd004-686d-4679-aca7-8d8d1b930029 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:17.972960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:17.972960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:17.972960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:17.972960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:17.972960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:17.972960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:17.972960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:17.972960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:17.972960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:17.972960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:17.972960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:17.972960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:17.972960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:17.972960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:17.973874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:17.973874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:17.973874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:17.973874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:17.973874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:17.973874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:17.973874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:17.973874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:17.973874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:17.973874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:17.973874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:17.973874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:17.973874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:17.973874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:17.973874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:17.973874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:17.973874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:17.974819 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:17.974819 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:17.974819 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:17.974819 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:17.974819 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:17.974819 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:17.974819 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:17.974819 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:17.978264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 309/604] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:18:17.979703 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08879abc-5b56-447c-8fbc-92502193a225 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:17.980185 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08879abc-5b56-447c-8fbc-92502193a225 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:17.980667 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08879abc-5b56-447c-8fbc-92502193a225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:18.003377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-08879abc-5b56-447c-8fbc-92502193a225 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:18.003566 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bfae2-084c-4c58-917c-ea5c132a29c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:18.004104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bfae2-084c-4c58-917c-ea5c132a29c2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:18.004391 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bfae2-084c-4c58-917c-ea5c132a29c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:18.004671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bfae2-084c-4c58-917c-ea5c132a29c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:18.005427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bfae2-084c-4c58-917c-ea5c132a29c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:18.005742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bfae2-084c-4c58-917c-ea5c132a29c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:18.006132 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-08879abc-5b56-447c-8fbc-92502193a225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:18.007384 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bfae2-084c-4c58-917c-ea5c132a29c2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:18.026228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 296/605] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:18.027178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bfae2-084c-4c58-917c-ea5c132a29c2 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:18.064346 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:18.090880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 310/606] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:18:18.092035 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08879abc-5b56-447c-8fbc-92502193a225 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:18.098513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:18.149986 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bfae2-084c-4c58-917c-ea5c132a29c2 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:18.186932 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be79965f-90f4-4fae-911b-391dd8d9cadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:18.187742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79965f-90f4-4fae-911b-391dd8d9cadb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:18.188112 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79965f-90f4-4fae-911b-391dd8d9cadb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:18.188646 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79965f-90f4-4fae-911b-391dd8d9cadb None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:18.189441 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be79965f-90f4-4fae-911b-391dd8d9cadb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:18.190348 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be79965f-90f4-4fae-911b-391dd8d9cadb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:18.192188 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be79965f-90f4-4fae-911b-391dd8d9cadb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:18.207530 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 297/607] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:18.209301 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a64d71-c23b-417c-9c69-312d9004137c None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:18.209962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a64d71-c23b-417c-9c69-312d9004137c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:18.211721 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a64d71-c23b-417c-9c69-312d9004137c None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:18.220822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 298/608] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:18.221685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a64d71-c23b-417c-9c69-312d9004137c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:18.237639 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08879abc-5b56-447c-8fbc-92502193a225 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:18.284432 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624d97a-a6d7-408e-8cbe-718c7c23ffcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:18.285090 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624d97a-a6d7-408e-8cbe-718c7c23ffcb None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:18.285370 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624d97a-a6d7-408e-8cbe-718c7c23ffcb None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:18.285646 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624d97a-a6d7-408e-8cbe-718c7c23ffcb None admin] PATH_INFO: `/v3/projects/479530ced580426b85d1fc07af43417f/users/84d4bd0a456341f78b5e476c97a9165c/roles/b0923c7a240149ae9ece5b3c1622e860` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:18.308649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9624d97a-a6d7-408e-8cbe-718c7c23ffcb None admin] RBAC: Authorizing `identity:create_grant(project_id=479530ced580426b85d1fc07af43417f, user_id=84d4bd0a456341f78b5e476c97a9165c, role_id=b0923c7a240149ae9ece5b3c1622e860)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:18.310833 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9624d97a-a6d7-408e-8cbe-718c7c23ffcb None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:18.341009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 311/609] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Sep 1 16:18:18 2022] PUT /identity/v3/projects/479530ced580426b85d1fc07af43417f/users/84d4bd0a456341f78b5e476c97a9165c/roles/b0923c7a240149ae9ece5b3c1622e860 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:18:18.342519 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c00021-a772-4b10-81eb-7973b97b5256 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:18.343026 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c00021-a772-4b10-81eb-7973b97b5256 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:18.343500 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c00021-a772-4b10-81eb-7973b97b5256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:18.360041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a64d71-c23b-417c-9c69-312d9004137c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:18.369972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a5c00021-a772-4b10-81eb-7973b97b5256 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:18.371788 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a5c00021-a772-4b10-81eb-7973b97b5256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:18.397331 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:18.408014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:18.432113 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a044a8-e323-4e14-aa73-5fa0c2cebba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:18.432782 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a044a8-e323-4e14-aa73-5fa0c2cebba3 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:18.433079 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a044a8-e323-4e14-aa73-5fa0c2cebba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:18.433359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a044a8-e323-4e14-aa73-5fa0c2cebba3 None admin] PATH_INFO: `/v3/projects/479530ced580426b85d1fc07af43417f/users/943fae29c5dc41f2911bbe051c74a361/roles/b0923c7a240149ae9ece5b3c1622e860` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:18.451082 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a044a8-e323-4e14-aa73-5fa0c2cebba3 None admin] RBAC: Authorizing `identity:create_grant(project_id=479530ced580426b85d1fc07af43417f, user_id=943fae29c5dc41f2911bbe051c74a361, role_id=b0923c7a240149ae9ece5b3c1622e860)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:18.452553 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a044a8-e323-4e14-aa73-5fa0c2cebba3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:18.470492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 299/610] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Sep 1 16:18:18 2022] PUT /identity/v3/projects/479530ced580426b85d1fc07af43417f/users/943fae29c5dc41f2911bbe051c74a361/roles/b0923c7a240149ae9ece5b3c1622e860 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:18:18.471583 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a044a8-e323-4e14-aa73-5fa0c2cebba3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:18.478117 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 312/611] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:18:18.588509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a044a8-e323-4e14-aa73-5fa0c2cebba3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:18.627204 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806f72c-4e68-4939-a952-3512d09c85cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:18.627852 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806f72c-4e68-4939-a952-3512d09c85cb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:18.628132 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806f72c-4e68-4939-a952-3512d09c85cb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:18.628407 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806f72c-4e68-4939-a952-3512d09c85cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:18.641156 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b806f72c-4e68-4939-a952-3512d09c85cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:18.646258 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-b806f72c-4e68-4939-a952-3512d09c85cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:18.646258 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:18.646258 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:18.646258 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:18.646258 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:18.646258 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:18.646258 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:18.646258 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:18.646258 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:18.646258 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:18.646258 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:18.646258 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:18.646258 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:18.646258 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:18.646258 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:18.646258 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:18.647323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:18.647323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:18.647323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:18.647323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:18.647323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:18.647323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:18.647323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:18.647323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:18.647323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:18.647323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:18.647323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:18.647323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:18.647323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:18.647323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:18.647323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:18.647323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:18.648196 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:18.648196 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:18.648196 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:18.648196 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:18.648196 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:18.648196 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:18.648196 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:18.648196 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:18.648196 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:18.648196 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:18.648196 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:18.648196 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:18.648196 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 300/612] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Sep 1 16:18:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:18.656119 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c00021-a772-4b10-81eb-7973b97b5256 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:18.771150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c00021-a772-4b10-81eb-7973b97b5256 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:18.799249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806f72c-4e68-4939-a952-3512d09c85cb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:18.810446 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8786b-58af-4975-92e1-0bbf65010a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:18.811061 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8786b-58af-4975-92e1-0bbf65010a0a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:18.811473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8786b-58af-4975-92e1-0bbf65010a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:18.811749 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8786b-58af-4975-92e1-0bbf65010a0a None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:18.812427 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8786b-58af-4975-92e1-0bbf65010a0a None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:18.812726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8786b-58af-4975-92e1-0bbf65010a0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:18.814325 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8786b-58af-4975-92e1-0bbf65010a0a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:18.824237 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 313/613] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Sep 1 16:18:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:18.829554 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8786b-58af-4975-92e1-0bbf65010a0a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:18.832611 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:18.850369 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:18.913567 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806f72c-4e68-4939-a952-3512d09c85cb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:18.935227 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8786b-58af-4975-92e1-0bbf65010a0a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:18.952253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd8f73-7d0c-4a94-8e52-64badd4cb5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:18.953242 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd8f73-7d0c-4a94-8e52-64badd4cb5f9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:18.953586 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd8f73-7d0c-4a94-8e52-64badd4cb5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:18.954317 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd8f73-7d0c-4a94-8e52-64badd4cb5f9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:18.965949 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd8f73-7d0c-4a94-8e52-64badd4cb5f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:18.970973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-1fdd8f73-7d0c-4a94-8e52-64badd4cb5f9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:18.970973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:18.970973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:18.970973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:18.970973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:18.970973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:18.970973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:18.970973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:18.970973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:18.970973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:18.970973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:18.970973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:18.970973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:18.970973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:18.970973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:18.970973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:18.971963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:18.971963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:18.971963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:18.971963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:18.971963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:18.971963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:18.971963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:18.971963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:18.971963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:18.971963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:18.971963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:18.971963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:18.971963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:18.971963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:18.971963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:18.971963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:18.972821 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:18.972821 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:18.972821 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:18.972821 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:18.972821 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:18.972821 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:18.972821 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:18.972821 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:18.972821 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:18.972821 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:18.972821 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:18.972821 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:18.972821 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 301/614] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Sep 1 16:18:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:18.978349 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd8f73-7d0c-4a94-8e52-64badd4cb5f9 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:19.005146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61198501-9194-4fc9-805e-8a77dc58d4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:19.005837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61198501-9194-4fc9-805e-8a77dc58d4b6 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:19.006124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61198501-9194-4fc9-805e-8a77dc58d4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:19.006476 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61198501-9194-4fc9-805e-8a77dc58d4b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:19.026971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61198501-9194-4fc9-805e-8a77dc58d4b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:19.028338 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61198501-9194-4fc9-805e-8a77dc58d4b6 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:19.058251 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-61198501-9194-4fc9-805e-8a77dc58d4b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:19.058251 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:19.058251 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:19.058251 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:19.058251 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:19.058251 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:19.058251 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:19.058251 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:19.058251 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:19.058251 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:19.058251 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:19.058251 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:19.058251 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:19.058251 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:19.058251 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:19.058251 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:19.059245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:19.059245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:19.059245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:19.059245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:19.059245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:19.059245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:19.059245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:19.059245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:19.059245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:19.059245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:19.059245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:19.059245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:19.059245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:19.059245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:19.059245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:19.059245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:19.059245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:19.060188 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:19.060188 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:19.060188 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:19.060188 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 314/615] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:18:19.071206 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61198501-9194-4fc9-805e-8a77dc58d4b6 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:19.129284 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:19.148875 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:19.163961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd8f73-7d0c-4a94-8e52-64badd4cb5f9 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:19.182490 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61198501-9194-4fc9-805e-8a77dc58d4b6 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:19.219615 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149500a-fb8f-48c5-9254-909cdf92ed44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:19.220965 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149500a-fb8f-48c5-9254-909cdf92ed44 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:19.222313 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149500a-fb8f-48c5-9254-909cdf92ed44 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:19.222769 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149500a-fb8f-48c5-9254-909cdf92ed44 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:19.224084 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149500a-fb8f-48c5-9254-909cdf92ed44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:19.225006 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149500a-fb8f-48c5-9254-909cdf92ed44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:19.227392 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149500a-fb8f-48c5-9254-909cdf92ed44 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:19.228728 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e2f7b-ae8a-4ad9-924e-a49b3a2022cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:19.229534 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e2f7b-ae8a-4ad9-924e-a49b3a2022cb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:19.229991 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e2f7b-ae8a-4ad9-924e-a49b3a2022cb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:19.230476 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e2f7b-ae8a-4ad9-924e-a49b3a2022cb None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:19.232939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e2f7b-ae8a-4ad9-924e-a49b3a2022cb None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:19.233435 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e2f7b-ae8a-4ad9-924e-a49b3a2022cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:19.236379 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e2f7b-ae8a-4ad9-924e-a49b3a2022cb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:19.251855 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 315/616] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:19.255251 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 302/617] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Sep 1 16:18:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:18:19.260677 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149500a-fb8f-48c5-9254-909cdf92ed44 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:19.261705 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e2f7b-ae8a-4ad9-924e-a49b3a2022cb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:19.449597 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149500a-fb8f-48c5-9254-909cdf92ed44 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:19.454523 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e2f7b-ae8a-4ad9-924e-a49b3a2022cb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:19.497219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c317224-ab53-4308-a25a-160edb8d96d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:19.497856 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c317224-ab53-4308-a25a-160edb8d96d7 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:19.498138 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c317224-ab53-4308-a25a-160edb8d96d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:19.498461 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c317224-ab53-4308-a25a-160edb8d96d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:19.510905 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c317224-ab53-4308-a25a-160edb8d96d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:19.512735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c317224-ab53-4308-a25a-160edb8d96d7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:19.528525 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-0c317224-ab53-4308-a25a-160edb8d96d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:19.528525 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:19.528525 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:19.528525 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:19.528525 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:19.528525 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:19.528525 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:19.528525 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:19.528525 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:19.528525 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:19.528525 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:19.528525 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:19.528525 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:19.528525 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:19.528525 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:19.528525 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:19.529588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:19.529588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:19.529588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:19.529588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:19.529588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:19.529588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:19.529588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:19.529588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:19.529588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:19.529588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:19.529588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:19.529588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:19.529588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:19.529588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:19.529588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:19.529588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:19.529588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:19.530529 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:19.530529 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:19.530529 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:19.530529 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 303/618] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:19.530771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:19.536209 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c317224-ab53-4308-a25a-160edb8d96d7 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:19.547904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:19.581066 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ebf989-a598-44e8-a103-a75d1200d3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:19.581756 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebf989-a598-44e8-a103-a75d1200d3b3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:19.582036 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebf989-a598-44e8-a103-a75d1200d3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:19.582335 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebf989-a598-44e8-a103-a75d1200d3b3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:19.599022 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ebf989-a598-44e8-a103-a75d1200d3b3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:19.600476 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ebf989-a598-44e8-a103-a75d1200d3b3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:19.616649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-d3ebf989-a598-44e8-a103-a75d1200d3b3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:18:19.616649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:19.616649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:19.616649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:19.616649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:19.616649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:19.616649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:19.616649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:19.616649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:19.616649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:19.616649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:19.616649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:19.616649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:19.616649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:19.616649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:19.616649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:19.617701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:19.617701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:19.617701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:19.617701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:19.617701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:19.617701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:19.617701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:19.617701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:19.617701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:19.617701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:19.617701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:19.617701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:19.617701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:19.617701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:19.617701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:19.617701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:19.617701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:19.617701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:19.618969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:19.618969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:19.618969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:19.618969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:19.618969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:19.618969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:19.618969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:19.618969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:19.618969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:18:19.618969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:19.618969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 316/619] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Sep 1 16:18:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:19.619590 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018609d2-d5e3-4b40-a011-0232229644c3 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:19.619590 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018609d2-d5e3-4b40-a011-0232229644c3 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:19.619590 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018609d2-d5e3-4b40-a011-0232229644c3 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:19.621628 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 317/620] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:19.623620 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018609d2-d5e3-4b40-a011-0232229644c3 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:19.672482 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c317224-ab53-4308-a25a-160edb8d96d7 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:19.718600 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112927d8-4508-43da-923f-c865c870a50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:19.719123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112927d8-4508-43da-923f-c865c870a50e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:19.719402 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112927d8-4508-43da-923f-c865c870a50e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:19.719673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112927d8-4508-43da-923f-c865c870a50e None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:19.720323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112927d8-4508-43da-923f-c865c870a50e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:19.721154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112927d8-4508-43da-923f-c865c870a50e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:19.722842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112927d8-4508-43da-923f-c865c870a50e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:19.738093 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018609d2-d5e3-4b40-a011-0232229644c3 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:19.740657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 304/621] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:19.742085 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde89cdb-334f-4d56-8532-955a239d9a87 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:19.743146 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde89cdb-334f-4d56-8532-955a239d9a87 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:19.743622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde89cdb-334f-4d56-8532-955a239d9a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:19.762534 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fde89cdb-334f-4d56-8532-955a239d9a87 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:19.764053 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fde89cdb-334f-4d56-8532-955a239d9a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:19.781835 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce5980-9fad-4c84-a190-e01ae804d3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:19.782618 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce5980-9fad-4c84-a190-e01ae804d3cb None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:19.782922 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce5980-9fad-4c84-a190-e01ae804d3cb None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:19.783441 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce5980-9fad-4c84-a190-e01ae804d3cb None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:19.784056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ce5980-9fad-4c84-a190-e01ae804d3cb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:19.784359 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ce5980-9fad-4c84-a190-e01ae804d3cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:19.785832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ce5980-9fad-4c84-a190-e01ae804d3cb None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:19.823756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:19.829050 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 318/622] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Sep 1 16:18:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:19.829943 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce5980-9fad-4c84-a190-e01ae804d3cb None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:19.836362 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 305/623] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:18:19.837537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde89cdb-334f-4d56-8532-955a239d9a87 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:19.884847 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:19.943456 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce5980-9fad-4c84-a190-e01ae804d3cb None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:19.947800 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde89cdb-334f-4d56-8532-955a239d9a87 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:19.981667 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283aec5-94c9-433c-90a6-82a672e9247a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:19.982319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283aec5-94c9-433c-90a6-82a672e9247a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:19.982615 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283aec5-94c9-433c-90a6-82a672e9247a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:19.982943 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283aec5-94c9-433c-90a6-82a672e9247a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:19.985088 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da6137-de54-472c-b365-5c23e62c2af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:19.985938 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da6137-de54-472c-b365-5c23e62c2af9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:19.986514 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da6137-de54-472c-b365-5c23e62c2af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:19.986997 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da6137-de54-472c-b365-5c23e62c2af9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:19.991754 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a283aec5-94c9-433c-90a6-82a672e9247a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:19.993148 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a283aec5-94c9-433c-90a6-82a672e9247a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:20.005918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-a283aec5-94c9-433c-90a6-82a672e9247a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:18:20.005918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:20.005918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:20.005918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:20.005918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:20.005918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:20.005918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:20.005918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:20.005918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:20.005918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:20.005918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:20.005918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:20.005918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:20.005918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:20.005918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:20.005918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:20.006971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:20.006971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:20.006971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:20.006971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:20.006971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:20.006971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:20.006971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:20.006971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:20.006971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:20.006971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:20.006971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:20.006971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:20.006971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:20.006971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:20.006971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:20.006971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:20.006971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:20.006971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:20.008009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:20.008009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:20.008009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:20.008009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:20.008009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:20.008009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:20.008009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:20.008009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:20.008009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:18:20.008009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:20.008009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 319/624] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:18:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:20.012886 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283aec5-94c9-433c-90a6-82a672e9247a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:20.056631 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:20.067774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da6137-de54-472c-b365-5c23e62c2af9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:20.069595 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da6137-de54-472c-b365-5c23e62c2af9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:20.135155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283aec5-94c9-433c-90a6-82a672e9247a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:20.145062 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:20.174810 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 306/625] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:18:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:18:20.175420 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810632e-ea53-44d8-bd85-0decc87c5559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:20.176037 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810632e-ea53-44d8-bd85-0decc87c5559 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:20.176312 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810632e-ea53-44d8-bd85-0decc87c5559 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:20.176579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810632e-ea53-44d8-bd85-0decc87c5559 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:20.177325 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a810632e-ea53-44d8-bd85-0decc87c5559 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:20.177630 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a810632e-ea53-44d8-bd85-0decc87c5559 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:20.186227 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a810632e-ea53-44d8-bd85-0decc87c5559 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:20.188324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da6137-de54-472c-b365-5c23e62c2af9 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:20.206232 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 320/626] 10.222.0.58 () {56 vars in 1135 bytes} [Thu Sep 1 16:18:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:18:20.206232 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810632e-ea53-44d8-bd85-0decc87c5559 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:20.322305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810632e-ea53-44d8-bd85-0decc87c5559 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:20.348932 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:20.355880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdd0dd-e9ca-422e-8c4c-ac1ba918799f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:20.362215 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdd0dd-e9ca-422e-8c4c-ac1ba918799f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:20.362215 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdd0dd-e9ca-422e-8c4c-ac1ba918799f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:20.362215 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdd0dd-e9ca-422e-8c4c-ac1ba918799f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:20.369438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da6137-de54-472c-b365-5c23e62c2af9 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:20.384659 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fdd0dd-e9ca-422e-8c4c-ac1ba918799f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:20.386116 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fdd0dd-e9ca-422e-8c4c-ac1ba918799f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:20.401322 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-95fdd0dd-e9ca-422e-8c4c-ac1ba918799f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:20.401322 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:20.401322 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:20.401322 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:20.401322 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:20.401322 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:20.401322 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:20.401322 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:20.401322 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:20.401322 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:20.401322 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:20.401322 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:20.401322 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:20.401322 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:20.401322 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:20.401322 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:20.402390 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:20.402390 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:20.402390 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:20.402390 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:20.402390 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:20.402390 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:20.402390 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:20.402390 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:20.402390 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:20.402390 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:20.402390 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:20.402390 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:20.402390 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:20.402390 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:20.402390 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:20.402390 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:20.402390 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:20.403320 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:20.403320 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:20.403320 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:20.403320 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 321/627] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:20.404102 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdd0dd-e9ca-422e-8c4c-ac1ba918799f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:20.414216 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc5792-8cb8-4fb0-b1c8-ca798954db0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:20.414216 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc5792-8cb8-4fb0-b1c8-ca798954db0e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:20.414216 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc5792-8cb8-4fb0-b1c8-ca798954db0e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:20.414216 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc5792-8cb8-4fb0-b1c8-ca798954db0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:20.424276 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:20.428526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc5792-8cb8-4fb0-b1c8-ca798954db0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:20.431436 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc5792-8cb8-4fb0-b1c8-ca798954db0e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:20.450074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-87dc5792-8cb8-4fb0-b1c8-ca798954db0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:20.450074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:20.450074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:20.450074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:20.450074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:20.450074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:20.450074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:20.450074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:20.450074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:20.450074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:20.450074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:20.450074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:20.450074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:20.450074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:20.450074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:20.450074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:20.451107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:20.451107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:20.451107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:20.451107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:20.451107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:20.451107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:20.451107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:20.451107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:20.451107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:20.451107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:20.451107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:20.451107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:20.451107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:20.451107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:20.451107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:20.451107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:20.451107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:20.452050 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:20.452050 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:20.452050 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:20.452050 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 307/628] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:20.452283 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc5792-8cb8-4fb0-b1c8-ca798954db0e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:20.517207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdd0dd-e9ca-422e-8c4c-ac1ba918799f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:20.560134 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d80aa9-f925-4323-ad3c-686638e264e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:20.560773 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d80aa9-f925-4323-ad3c-686638e264e3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:20.561049 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d80aa9-f925-4323-ad3c-686638e264e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:20.561324 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d80aa9-f925-4323-ad3c-686638e264e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:20.572475 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d80aa9-f925-4323-ad3c-686638e264e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:20.577523 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-09d80aa9-f925-4323-ad3c-686638e264e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:20.577523 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:20.577523 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:20.577523 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:20.577523 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:20.577523 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:20.577523 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:20.577523 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:20.577523 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:20.577523 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:20.577523 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:20.577523 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:20.577523 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:20.577523 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:20.577523 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:20.577523 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:20.578508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:20.578508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:20.578508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:20.578508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:20.578508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:20.578508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:20.578508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:20.578508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:20.578508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:20.578508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:20.578508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:20.578508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:20.578508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:20.578508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:20.578508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:20.578508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:20.583416 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:20.583416 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:20.583416 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:20.583416 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:20.583416 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:20.583416 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:20.583416 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:20.583416 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:20.583416 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:20.583416 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:20.583416 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:20.583416 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:20.584099 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc5792-8cb8-4fb0-b1c8-ca798954db0e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:20.584186 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 322/629] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Sep 1 16:18:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:20.584186 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d80aa9-f925-4323-ad3c-686638e264e3 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:20.618377 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61488295-e5cf-4ec7-bf4d-01383cf073ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:20.618988 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61488295-e5cf-4ec7-bf4d-01383cf073ed None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:20.619275 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61488295-e5cf-4ec7-bf4d-01383cf073ed None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:20.619552 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61488295-e5cf-4ec7-bf4d-01383cf073ed None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:20.620200 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61488295-e5cf-4ec7-bf4d-01383cf073ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:20.620508 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61488295-e5cf-4ec7-bf4d-01383cf073ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:20.621894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61488295-e5cf-4ec7-bf4d-01383cf073ed None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:20.634585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 308/630] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:20.635855 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc0bf6-0a92-40f2-ab1b-06647ab939ac None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:20.636149 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc0bf6-0a92-40f2-ab1b-06647ab939ac None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:20.636425 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc0bf6-0a92-40f2-ab1b-06647ab939ac None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:20.637769 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 309/631] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:20.642611 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc0bf6-0a92-40f2-ab1b-06647ab939ac None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:20.645502 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:20.676140 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:20.700107 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d80aa9-f925-4323-ad3c-686638e264e3 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:20.734754 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80628d76-99e1-4e0d-8d6c-64e8a0ec2d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:20.735363 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80628d76-99e1-4e0d-8d6c-64e8a0ec2d4d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:20.735644 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80628d76-99e1-4e0d-8d6c-64e8a0ec2d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:20.735916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80628d76-99e1-4e0d-8d6c-64e8a0ec2d4d None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:20.736553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80628d76-99e1-4e0d-8d6c-64e8a0ec2d4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:20.736860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80628d76-99e1-4e0d-8d6c-64e8a0ec2d4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:20.738228 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80628d76-99e1-4e0d-8d6c-64e8a0ec2d4d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:20.755486 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 323/632] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:20.755486 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80628d76-99e1-4e0d-8d6c-64e8a0ec2d4d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:20.758797 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc0bf6-0a92-40f2-ab1b-06647ab939ac None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:20.797311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f263f4-56ad-4e0a-850d-396f976c2730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:20.798222 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f263f4-56ad-4e0a-850d-396f976c2730 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:20.798531 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f263f4-56ad-4e0a-850d-396f976c2730 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:20.798815 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f263f4-56ad-4e0a-850d-396f976c2730 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:20.799485 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f263f4-56ad-4e0a-850d-396f976c2730 None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:20.799792 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f263f4-56ad-4e0a-850d-396f976c2730 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:20.801164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f263f4-56ad-4e0a-850d-396f976c2730 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:20.814245 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 310/633] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Sep 1 16:18:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:20.814245 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b0056-10db-4d77-96de-1c78981b57cc None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:20.814245 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b0056-10db-4d77-96de-1c78981b57cc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:20.814245 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b0056-10db-4d77-96de-1c78981b57cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:20.829057 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-153b0056-10db-4d77-96de-1c78981b57cc None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:20.830211 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-153b0056-10db-4d77-96de-1c78981b57cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:20.862836 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80628d76-99e1-4e0d-8d6c-64e8a0ec2d4d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:20.894540 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:20.899644 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e90633-6613-4af9-98d2-e689fe611975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:20.900365 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e90633-6613-4af9-98d2-e689fe611975 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:20.900538 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e90633-6613-4af9-98d2-e689fe611975 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:20.900810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e90633-6613-4af9-98d2-e689fe611975 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:20.904536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 311/634] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:18:20.905174 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b0056-10db-4d77-96de-1c78981b57cc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:20.927657 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e90633-6613-4af9-98d2-e689fe611975 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:20.929230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e90633-6613-4af9-98d2-e689fe611975 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:20.932082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:20.953363 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-19e90633-6613-4af9-98d2-e689fe611975 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:20.953363 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:20.953363 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:20.953363 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:20.953363 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:20.953363 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:20.953363 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:20.953363 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:20.953363 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:20.953363 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:20.953363 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:20.953363 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:20.953363 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:20.953363 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:20.953363 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:20.954367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:20.954367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:20.954367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:20.954367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:20.954367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:20.954367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:20.954367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:20.954367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:20.954367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:20.954367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:20.954367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:20.954367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:20.954367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:20.954367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:20.954367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:20.954367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:20.954367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:20.955272 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:20.955272 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:20.955272 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:20.955272 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:20.955272 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:20.955272 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:20.955272 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:20.955272 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:20.958348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 324/635] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:20.959586 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e90633-6613-4af9-98d2-e689fe611975 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:21.021939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b0056-10db-4d77-96de-1c78981b57cc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:21.070958 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183b97eb-351a-4da5-88ef-52e205acc213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:21.071592 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b97eb-351a-4da5-88ef-52e205acc213 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:21.071873 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b97eb-351a-4da5-88ef-52e205acc213 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:21.072145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b97eb-351a-4da5-88ef-52e205acc213 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:21.081728 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183b97eb-351a-4da5-88ef-52e205acc213 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:21.083139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183b97eb-351a-4da5-88ef-52e205acc213 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:21.097627 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e90633-6613-4af9-98d2-e689fe611975 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:21.098650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-183b97eb-351a-4da5-88ef-52e205acc213 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:21.098650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:21.098650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:21.098650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:21.098650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:21.098650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:21.098650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:21.098650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:21.098650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:21.098650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:21.098650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:21.098650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:21.098650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:21.098650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:21.098650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:21.099756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:21.099756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:21.099756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:21.099756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:21.099756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:21.099756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:21.099756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:21.099756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:21.099756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:21.099756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:21.099756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:21.099756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:21.099756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:21.099756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:21.099756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:21.099756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:21.099756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:21.100867 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:21.100867 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:21.100867 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:21.100867 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:21.100867 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:21.100867 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:21.100867 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:21.100867 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:21.105244 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 312/636] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:21.107163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183b97eb-351a-4da5-88ef-52e205acc213 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:21.140573 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc6289-2f03-43f9-9ac9-8ded6647cc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:21.141206 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc6289-2f03-43f9-9ac9-8ded6647cc04 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:21.141570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc6289-2f03-43f9-9ac9-8ded6647cc04 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:21.141812 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc6289-2f03-43f9-9ac9-8ded6647cc04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:21.153232 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc6289-2f03-43f9-9ac9-8ded6647cc04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:21.155009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc6289-2f03-43f9-9ac9-8ded6647cc04 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:21.168843 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:21.172766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-51fc6289-2f03-43f9-9ac9-8ded6647cc04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:21.172766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:21.172766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:21.172766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:21.172766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:21.172766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:21.172766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:21.172766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:21.172766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:21.172766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:21.172766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:21.172766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:21.172766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:21.172766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:21.172766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:21.172766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:21.173898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:21.173898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:21.173898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:21.173898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:21.173898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:21.173898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:21.173898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:21.173898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:21.173898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:21.173898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:21.173898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:21.173898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:21.173898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:21.173898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:21.173898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:21.173898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:21.173898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:21.175060 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:21.175060 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:21.175060 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:21.176404 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 325/637] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:21.177442 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc6289-2f03-43f9-9ac9-8ded6647cc04 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:21.221677 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183b97eb-351a-4da5-88ef-52e205acc213 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:21.226105 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:21.256206 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6e223-47d9-4956-8df5-a95992b96c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:21.256912 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6e223-47d9-4956-8df5-a95992b96c60 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:21.257197 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6e223-47d9-4956-8df5-a95992b96c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:21.257471 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6e223-47d9-4956-8df5-a95992b96c60 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:21.258757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e6e223-47d9-4956-8df5-a95992b96c60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:21.259059 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e6e223-47d9-4956-8df5-a95992b96c60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:21.260482 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e6e223-47d9-4956-8df5-a95992b96c60 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:21.271658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 313/638] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:21.272788 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7314da-410d-4bcc-8e9e-d455268b6b9e None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:21.273076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7314da-410d-4bcc-8e9e-d455268b6b9e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:21.273352 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7314da-410d-4bcc-8e9e-d455268b6b9e None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:21.274867 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 314/639] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:21.275570 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7314da-410d-4bcc-8e9e-d455268b6b9e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:21.281175 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc6289-2f03-43f9-9ac9-8ded6647cc04 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:21.315800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e6c67-7d1c-41ee-93a3-07ccd74e5f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:21.316593 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e6c67-7d1c-41ee-93a3-07ccd74e5f12 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:21.316869 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e6c67-7d1c-41ee-93a3-07ccd74e5f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:21.317133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e6c67-7d1c-41ee-93a3-07ccd74e5f12 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:21.317809 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e6c67-7d1c-41ee-93a3-07ccd74e5f12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:21.318108 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e6c67-7d1c-41ee-93a3-07ccd74e5f12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:21.319543 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e6c67-7d1c-41ee-93a3-07ccd74e5f12 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:21.330952 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 326/640] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:21.331788 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e6c67-7d1c-41ee-93a3-07ccd74e5f12 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:21.379696 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7314da-410d-4bcc-8e9e-d455268b6b9e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:21.393138 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:21.412918 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa2882-e54b-4998-89bf-036c515c00b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:21.413516 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa2882-e54b-4998-89bf-036c515c00b5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:21.413795 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa2882-e54b-4998-89bf-036c515c00b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:21.414066 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa2882-e54b-4998-89bf-036c515c00b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:21.414753 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa2882-e54b-4998-89bf-036c515c00b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:21.415054 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa2882-e54b-4998-89bf-036c515c00b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:21.416402 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa2882-e54b-4998-89bf-036c515c00b5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:21.428587 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e6c67-7d1c-41ee-93a3-07ccd74e5f12 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:21.428733 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 315/641] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:21.429451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa2882-e54b-4998-89bf-036c515c00b5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:21.439220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:21.467853 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca004a88-7357-44f2-aa3c-5176ad351ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:21.468483 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca004a88-7357-44f2-aa3c-5176ad351ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:21.468767 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca004a88-7357-44f2-aa3c-5176ad351ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:21.469029 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca004a88-7357-44f2-aa3c-5176ad351ce1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:21.480660 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca004a88-7357-44f2-aa3c-5176ad351ce1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:21.485464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-ca004a88-7357-44f2-aa3c-5176ad351ce1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:21.485464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:21.485464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:21.485464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:21.485464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:21.485464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:21.485464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:21.485464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:21.485464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:21.485464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:21.485464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:21.485464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:21.485464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:21.485464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:21.485464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:21.485464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:21.486614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:21.486614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:21.486614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:21.486614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:21.486614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:21.486614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:21.486614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:21.486614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:21.486614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:21.486614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:21.486614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:21.486614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:21.486614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:21.486614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:21.486614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:21.486614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:21.487534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:21.487534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:21.487534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:21.487534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:21.487534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:21.487534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:21.487534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:21.487534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:21.487534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:21.487534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:21.487534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:21.487534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:21.487534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 327/642] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Sep 1 16:18:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:21.488349 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf977c7-c5cd-4f71-bffa-448367bb3271 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:21.488457 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf977c7-c5cd-4f71-bffa-448367bb3271 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:21.488733 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf977c7-c5cd-4f71-bffa-448367bb3271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:21.506509 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-edf977c7-c5cd-4f71-bffa-448367bb3271 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:21.507791 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-edf977c7-c5cd-4f71-bffa-448367bb3271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:21.536107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa2882-e54b-4998-89bf-036c515c00b5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:21.569283 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299be5a-8397-4def-8c6c-359b3613a7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:21.569933 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299be5a-8397-4def-8c6c-359b3613a7d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:21.570235 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299be5a-8397-4def-8c6c-359b3613a7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:21.571041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299be5a-8397-4def-8c6c-359b3613a7d3 None admin] PATH_INFO: `/v3/projects/479530ced580426b85d1fc07af43417f/users/e789695517c1437fb224f5a6195655f9/roles/b0923c7a240149ae9ece5b3c1622e860` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:21.579966 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 328/643] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:18:21.579966 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf977c7-c5cd-4f71-bffa-448367bb3271 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:21.589438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c299be5a-8397-4def-8c6c-359b3613a7d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=479530ced580426b85d1fc07af43417f, user_id=e789695517c1437fb224f5a6195655f9, role_id=b0923c7a240149ae9ece5b3c1622e860)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:21.590885 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c299be5a-8397-4def-8c6c-359b3613a7d3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:21.604667 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 316/644] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Sep 1 16:18:21 2022] PUT /identity/v3/projects/479530ced580426b85d1fc07af43417f/users/e789695517c1437fb224f5a6195655f9/roles/b0923c7a240149ae9ece5b3c1622e860 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:18:21.605307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299be5a-8397-4def-8c6c-359b3613a7d3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:21.631810 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:21.704758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:21.709020 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf977c7-c5cd-4f71-bffa-448367bb3271 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:21.749384 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299be5a-8397-4def-8c6c-359b3613a7d3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:21.749640 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c690ebd4-65d5-4baa-87c8-d2a6619474f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:21.750155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690ebd4-65d5-4baa-87c8-d2a6619474f8 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:21.750516 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690ebd4-65d5-4baa-87c8-d2a6619474f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:21.750802 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690ebd4-65d5-4baa-87c8-d2a6619474f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:21.751725 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c690ebd4-65d5-4baa-87c8-d2a6619474f8 None admin] RBAC: Adding query filter params (role.id=b0923c7a240149ae9ece5b3c1622e860, scope.project.id=479530ced580426b85d1fc07af43417f, user.id=1f0c99fdf8bd4331ad7e641ee38554c1) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:21.752041 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c690ebd4-65d5-4baa-87c8-d2a6619474f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:21.754395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c690ebd4-65d5-4baa-87c8-d2a6619474f8 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:21.767046 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 329/645] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:18:21 2022] GET /identity/v3/role_assignments?user.id=1f0c99fdf8bd4331ad7e641ee38554c1&scope.project.id=479530ced580426b85d1fc07af43417f&role.id=b0923c7a240149ae9ece5b3c1622e860 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:21.768634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c690ebd4-65d5-4baa-87c8-d2a6619474f8 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:21.794919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc45cc-7d95-4630-9924-89f42fc31c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:21.796297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc45cc-7d95-4630-9924-89f42fc31c66 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:21.796778 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc45cc-7d95-4630-9924-89f42fc31c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:21.797329 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc45cc-7d95-4630-9924-89f42fc31c66 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:21.807046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc45cc-7d95-4630-9924-89f42fc31c66 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:21.808916 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc45cc-7d95-4630-9924-89f42fc31c66 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:21.824429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-abbc45cc-7d95-4630-9924-89f42fc31c66 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 16:18:21.824429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:21.824429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:21.824429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:21.824429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:21.824429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:21.824429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:21.824429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:21.824429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:21.824429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:21.824429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:21.824429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:21.824429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:21.824429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:21.824429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:21.824429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:21.825588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:21.825588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:21.825588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:21.825588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:21.825588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:21.825588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:21.825588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:21.825588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:21.825588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:21.825588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:21.825588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:21.825588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:21.825588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:21.825588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:21.825588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:21.825588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:21.825588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:21.825588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:21.826690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:21.826690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:21.826690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:21.826690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:21.826690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:21.826690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:21.826690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:21.826690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:21.826690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 16:18:21.826690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:21.829432 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 317/646] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:18:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:21.834550 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc45cc-7d95-4630-9924-89f42fc31c66 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:21.898583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c690ebd4-65d5-4baa-87c8-d2a6619474f8 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:21.940047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f8a65-006a-4d60-a232-f61ab470bcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:21.940856 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f8a65-006a-4d60-a232-f61ab470bcfc None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:21.941197 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f8a65-006a-4d60-a232-f61ab470bcfc None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:21.941571 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f8a65-006a-4d60-a232-f61ab470bcfc None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:21.942473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f8a65-006a-4d60-a232-f61ab470bcfc None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:21.942855 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f8a65-006a-4d60-a232-f61ab470bcfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:21.944449 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f8a65-006a-4d60-a232-f61ab470bcfc None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:21.948650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:21.960394 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 330/647] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Sep 1 16:18:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:21.961406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f8a65-006a-4d60-a232-f61ab470bcfc None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:21.973961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc45cc-7d95-4630-9924-89f42fc31c66 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:21.996729 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:22.018319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb7e34-bffe-4ecb-ac90-90589a8bf7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:22.019680 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb7e34-bffe-4ecb-ac90-90589a8bf7da None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:22.019680 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb7e34-bffe-4ecb-ac90-90589a8bf7da None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:22.019680 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb7e34-bffe-4ecb-ac90-90589a8bf7da None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:22.020436 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb7e34-bffe-4ecb-ac90-90589a8bf7da None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:22.020745 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb7e34-bffe-4ecb-ac90-90589a8bf7da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:22.022367 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb7e34-bffe-4ecb-ac90-90589a8bf7da None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:22.050103 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 318/648] 10.222.0.58 () {56 vars in 1137 bytes} [Thu Sep 1 16:18:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:18:22.051038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb7e34-bffe-4ecb-ac90-90589a8bf7da None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:22.107661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f8a65-006a-4d60-a232-f61ab470bcfc None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:22.153785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c5f52d-77e4-4d94-a9b3-6f00aff783da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:22.155282 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5f52d-77e4-4d94-a9b3-6f00aff783da None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:22.155763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5f52d-77e4-4d94-a9b3-6f00aff783da None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:22.156481 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5f52d-77e4-4d94-a9b3-6f00aff783da None admin] PATH_INFO: `/v3/roles/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:22.170070 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c5f52d-77e4-4d94-a9b3-6f00aff783da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:22.170461 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb7e34-bffe-4ecb-ac90-90589a8bf7da None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:22.176348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-33c5f52d-77e4-4d94-a9b3-6f00aff783da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:22.176348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:22.176348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:22.176348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:22.176348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:22.176348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:22.176348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:22.176348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:22.176348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:22.176348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:22.176348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:22.176348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:22.176348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:22.176348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:22.176348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:22.176348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:22.177328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:22.177328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:22.177328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:22.177328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:22.177328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:22.177328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:22.177328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:22.177328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:22.177328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:22.177328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:22.177328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:22.177328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:22.177328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:22.177328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:22.177328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:22.177328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:22.178421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:22.178421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:22.178421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:22.178421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:22.178421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:22.178421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:22.178421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:22.178421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:22.178421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:22.178421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:22.178421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:22.178421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:22.178421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 331/649] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Sep 1 16:18:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:22.180565 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c5f52d-77e4-4d94-a9b3-6f00aff783da None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:22.214391 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd21e42-50fe-4c57-a7a2-08c9662cc38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:22.215454 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd21e42-50fe-4c57-a7a2-08c9662cc38a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:22.215896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd21e42-50fe-4c57-a7a2-08c9662cc38a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:22.216423 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd21e42-50fe-4c57-a7a2-08c9662cc38a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:22.228787 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd21e42-50fe-4c57-a7a2-08c9662cc38a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:22.230596 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd21e42-50fe-4c57-a7a2-08c9662cc38a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:22.232764 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:22.250491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-1cd21e42-50fe-4c57-a7a2-08c9662cc38a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:22.250491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:22.250491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:22.250491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:22.250491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:22.250491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:22.250491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:22.250491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:22.250491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:22.250491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:22.250491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:22.250491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:22.250491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:22.250491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:22.250491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:22.250491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:22.251735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:22.251735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:22.251735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:22.251735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:22.251735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:22.251735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:22.251735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:22.251735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:22.251735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:22.251735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:22.251735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:22.251735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:22.251735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:22.251735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:22.251735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:22.251735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:22.251735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:22.252664 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:22.252664 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:22.252664 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:22.254309 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 319/650] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:22.255438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd21e42-50fe-4c57-a7a2-08c9662cc38a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:22.273396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:22.294007 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c5f52d-77e4-4d94-a9b3-6f00aff783da None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:22.329206 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5518416-af54-4c91-9117-cf03594dbca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:22.329951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5518416-af54-4c91-9117-cf03594dbca5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:22.330181 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5518416-af54-4c91-9117-cf03594dbca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:22.330484 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5518416-af54-4c91-9117-cf03594dbca5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:22.385638 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd21e42-50fe-4c57-a7a2-08c9662cc38a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:22.401751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5518416-af54-4c91-9117-cf03594dbca5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:22.403177 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5518416-af54-4c91-9117-cf03594dbca5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:22.431184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce77048-4449-4b8e-84be-a12f43f8f90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:22.432494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce77048-4449-4b8e-84be-a12f43f8f90c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:22.432940 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce77048-4449-4b8e-84be-a12f43f8f90c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:22.433614 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce77048-4449-4b8e-84be-a12f43f8f90c None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:22.434644 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce77048-4449-4b8e-84be-a12f43f8f90c None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:22.435432 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce77048-4449-4b8e-84be-a12f43f8f90c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:22.437251 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce77048-4449-4b8e-84be-a12f43f8f90c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:22.450523 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 320/651] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Sep 1 16:18:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:22.451906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce77048-4449-4b8e-84be-a12f43f8f90c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:22.499480 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:22.516667 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:22.526577 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 332/652] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:18:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:18:22.527895 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5518416-af54-4c91-9117-cf03594dbca5 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:22.570372 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce77048-4449-4b8e-84be-a12f43f8f90c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:22.608432 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c4cbf-d0ff-4924-a2e1-7165e875faae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:22.609036 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c4cbf-d0ff-4924-a2e1-7165e875faae None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:22.609307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c4cbf-d0ff-4924-a2e1-7165e875faae None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:22.609578 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c4cbf-d0ff-4924-a2e1-7165e875faae None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:22.610234 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c4cbf-d0ff-4924-a2e1-7165e875faae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:22.610571 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c4cbf-d0ff-4924-a2e1-7165e875faae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:22.611988 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c4cbf-d0ff-4924-a2e1-7165e875faae None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:22.624444 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 321/653] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:22.625362 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c4cbf-d0ff-4924-a2e1-7165e875faae None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:22.663578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5518416-af54-4c91-9117-cf03594dbca5 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:22.701628 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1099ef-8c7b-42df-8db2-199860a946fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:22.702182 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1099ef-8c7b-42df-8db2-199860a946fe None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:22.702482 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1099ef-8c7b-42df-8db2-199860a946fe None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:22.702777 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1099ef-8c7b-42df-8db2-199860a946fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:22.713783 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1099ef-8c7b-42df-8db2-199860a946fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:22.715247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1099ef-8c7b-42df-8db2-199860a946fe None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:22.731579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-8e1099ef-8c7b-42df-8db2-199860a946fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:22.731579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:22.731579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:22.731579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:22.731579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:22.731579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:22.731579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:22.731579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:22.731579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:22.731579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:22.731579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:22.731579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:22.731579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:22.731579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:22.731579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:22.731579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:22.732778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:22.732778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:22.732778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:22.732778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:22.732778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:22.732778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:22.732778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:22.732778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:22.732778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:22.732778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:22.732778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:22.732778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:22.732778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:22.732778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:22.732778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:22.732778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:22.732778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:22.734278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:22.734278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:22.734278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:22.737374 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 333/654] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:22.738296 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1099ef-8c7b-42df-8db2-199860a946fe None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:22.747978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c4cbf-d0ff-4924-a2e1-7165e875faae None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:22.768079 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:22.784628 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:22.796389 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c6a6c-083e-4de6-a3e9-489736170792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:22.797570 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c6a6c-083e-4de6-a3e9-489736170792 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:22.798012 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c6a6c-083e-4de6-a3e9-489736170792 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:22.799485 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c6a6c-083e-4de6-a3e9-489736170792 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:22.813943 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c6a6c-083e-4de6-a3e9-489736170792 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:22.816368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c6a6c-083e-4de6-a3e9-489736170792 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:22.836909 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-e75c6a6c-083e-4de6-a3e9-489736170792 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:22.836909 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:22.836909 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:22.836909 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:22.836909 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:22.836909 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:22.836909 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:22.836909 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:22.836909 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:22.836909 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:22.836909 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:22.836909 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:22.836909 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:22.836909 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:22.836909 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:22.836909 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:22.838166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:22.838166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:22.838166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:22.838166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:22.838166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:22.838166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:22.838166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:22.838166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:22.838166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:22.838166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:22.838166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:22.838166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:22.838166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:22.838166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:22.838166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:22.838166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:22.838166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:22.839321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:22.839321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:22.839321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:22.842091 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 322/655] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:22.843443 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c6a6c-083e-4de6-a3e9-489736170792 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:22.853290 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1099ef-8c7b-42df-8db2-199860a946fe None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:22.889977 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4b749-cc03-4da8-8879-91bbee4cb870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:22.890734 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4b749-cc03-4da8-8879-91bbee4cb870 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:22.891088 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4b749-cc03-4da8-8879-91bbee4cb870 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:22.891434 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4b749-cc03-4da8-8879-91bbee4cb870 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:22.901355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4b749-cc03-4da8-8879-91bbee4cb870 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:22.903050 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4b749-cc03-4da8-8879-91bbee4cb870 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:22.917565 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-3ce4b749-cc03-4da8-8879-91bbee4cb870 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:18:22.917565 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:22.917565 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:22.917565 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:22.917565 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:22.917565 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:22.917565 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:22.917565 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:22.917565 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:22.917565 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:22.917565 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:22.917565 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:22.917565 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:22.917565 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:22.917565 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:22.917565 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:22.918574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:22.918574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:22.918574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:22.918574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:22.918574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:22.918574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:22.918574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:22.918574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:22.918574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:22.918574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:22.918574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:22.918574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:22.918574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:22.918574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:22.918574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:22.918574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:22.918574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:22.918574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:22.919553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:22.919553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:22.919553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:22.919553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:22.919553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:22.919553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:22.919553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:22.919553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:22.919553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:18:22.919553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:22.919553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 334/656] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:18:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:22.920178 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4b749-cc03-4da8-8879-91bbee4cb870 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:22.985605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c6a6c-083e-4de6-a3e9-489736170792 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:23.030549 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:23.036569 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4b749-cc03-4da8-8879-91bbee4cb870 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:23.039666 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1007f-7082-45a8-9a3d-a12d158dd36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:23.041447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1007f-7082-45a8-9a3d-a12d158dd36f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:23.041883 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1007f-7082-45a8-9a3d-a12d158dd36f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:23.042579 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1007f-7082-45a8-9a3d-a12d158dd36f None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:23.043413 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1007f-7082-45a8-9a3d-a12d158dd36f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:23.043870 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1007f-7082-45a8-9a3d-a12d158dd36f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:23.045419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1007f-7082-45a8-9a3d-a12d158dd36f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:23.064667 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 323/657] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:23.065818 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1007f-7082-45a8-9a3d-a12d158dd36f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:23.081334 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6fc74d-bdb8-46f1-9f0d-b7aa9fdb77a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:23.082137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fc74d-bdb8-46f1-9f0d-b7aa9fdb77a4 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:23.082530 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fc74d-bdb8-46f1-9f0d-b7aa9fdb77a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:23.082901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fc74d-bdb8-46f1-9f0d-b7aa9fdb77a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:23.083659 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6fc74d-bdb8-46f1-9f0d-b7aa9fdb77a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:23.084312 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6fc74d-bdb8-46f1-9f0d-b7aa9fdb77a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:23.085711 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6fc74d-bdb8-46f1-9f0d-b7aa9fdb77a4 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:23.099108 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 335/658] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:23.100054 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6fc74d-bdb8-46f1-9f0d-b7aa9fdb77a4 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:23.106073 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:23.203199 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1007f-7082-45a8-9a3d-a12d158dd36f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:23.212914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6fc74d-bdb8-46f1-9f0d-b7aa9fdb77a4 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:23.249926 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f3f93-7cf9-4445-b4d0-e595907d9a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:23.250832 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f3f93-7cf9-4445-b4d0-e595907d9a45 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:23.250951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb39f9a-4cd1-4d39-a3bc-6b887b2584e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:23.251401 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f3f93-7cf9-4445-b4d0-e595907d9a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:23.251767 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb39f9a-4cd1-4d39-a3bc-6b887b2584e1 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:23.251990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f3f93-7cf9-4445-b4d0-e595907d9a45 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:23.252223 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb39f9a-4cd1-4d39-a3bc-6b887b2584e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:23.252602 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb39f9a-4cd1-4d39-a3bc-6b887b2584e1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:23.253251 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818f3f93-7cf9-4445-b4d0-e595907d9a45 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:23.253798 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818f3f93-7cf9-4445-b4d0-e595907d9a45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:23.256285 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818f3f93-7cf9-4445-b4d0-e595907d9a45 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:23.262497 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb39f9a-4cd1-4d39-a3bc-6b887b2584e1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:23.264036 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb39f9a-4cd1-4d39-a3bc-6b887b2584e1 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:23.276865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-feb39f9a-4cd1-4d39-a3bc-6b887b2584e1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 16:18:23.276865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:23.276865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:23.276865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:23.276865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:23.276865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:23.276865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:23.276865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:23.276865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:23.276865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:23.276865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:23.276865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:23.276865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:23.276865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:23.276865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:23.276865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:23.278929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:23.278929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:23.278929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:23.278929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:23.278929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:23.278929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:23.278929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:23.278929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:23.278929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:23.278929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:23.278929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:23.278929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:23.278929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:23.278929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:23.278929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:23.278929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:23.278929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:23.278929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:23.280148 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:23.280148 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:23.280148 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:23.280148 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:23.280148 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:23.280148 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:23.280148 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:23.280148 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:23.280148 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 16:18:23.280148 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:23.285610 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 336/659] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Sep 1 16:18:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:23.286617 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb39f9a-4cd1-4d39-a3bc-6b887b2584e1 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:23.287874 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 324/660] 10.222.0.58 () {56 vars in 1137 bytes} [Thu Sep 1 16:18:23 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:23.288917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f3f93-7cf9-4445-b4d0-e595907d9a45 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:23.313583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:23.399251 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:23.406795 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb39f9a-4cd1-4d39-a3bc-6b887b2584e1 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:23.415815 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f3f93-7cf9-4445-b4d0-e595907d9a45 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:23.442562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c1335-8309-44c9-9a4a-70f10c273b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:23.443387 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c1335-8309-44c9-9a4a-70f10c273b29 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:23.443595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c1335-8309-44c9-9a4a-70f10c273b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:23.443879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c1335-8309-44c9-9a4a-70f10c273b29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:23.451836 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c1335-8309-44c9-9a4a-70f10c273b29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:23.453455 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c1335-8309-44c9-9a4a-70f10c273b29 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:23.456220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3588c19-4cc6-46d5-a609-c1e8ffb06fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:23.457058 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3588c19-4cc6-46d5-a609-c1e8ffb06fde None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:23.457532 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3588c19-4cc6-46d5-a609-c1e8ffb06fde None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:23.457988 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3588c19-4cc6-46d5-a609-c1e8ffb06fde None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:23.458931 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3588c19-4cc6-46d5-a609-c1e8ffb06fde None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:23.459421 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3588c19-4cc6-46d5-a609-c1e8ffb06fde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:23.461561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3588c19-4cc6-46d5-a609-c1e8ffb06fde None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:23.466401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-940c1335-8309-44c9-9a4a-70f10c273b29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:23.466401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:23.466401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:23.466401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:23.466401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:23.466401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:23.466401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:23.466401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:23.466401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:23.466401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:23.466401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:23.466401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:23.466401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:23.466401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:23.466401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:23.467323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:23.467323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:23.467323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:23.467323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:23.467323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:23.467323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:23.467323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:23.467323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:23.467323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:23.467323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:23.467323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:23.467323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:23.467323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:23.467323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:23.467323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:23.467323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:23.467323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:23.468399 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:23.468399 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:23.468399 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:23.468399 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:23.468399 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:23.468399 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:23.468399 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:23.468399 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:23.468399 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 337/661] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:23.469051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c1335-8309-44c9-9a4a-70f10c273b29 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:23.485582 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 325/662] 10.222.0.58 () {56 vars in 1139 bytes} [Thu Sep 1 16:18:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:23.486778 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3588c19-4cc6-46d5-a609-c1e8ffb06fde None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:23.561394 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:23.585910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c1335-8309-44c9-9a4a-70f10c273b29 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:23.602259 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3588c19-4cc6-46d5-a609-c1e8ffb06fde None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:23.623151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c475fc9-b6eb-479d-a8ff-d0883b67b4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:23.623853 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c475fc9-b6eb-479d-a8ff-d0883b67b4c3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:23.624340 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c475fc9-b6eb-479d-a8ff-d0883b67b4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:23.624618 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c475fc9-b6eb-479d-a8ff-d0883b67b4c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:23.647874 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da3522b-f491-4377-9a94-d8df19369cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:23.649408 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3522b-f491-4377-9a94-d8df19369cfe None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:23.649877 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3522b-f491-4377-9a94-d8df19369cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:23.650383 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3522b-f491-4377-9a94-d8df19369cfe None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:23.651260 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da3522b-f491-4377-9a94-d8df19369cfe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:23.651816 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da3522b-f491-4377-9a94-d8df19369cfe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:23.653552 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da3522b-f491-4377-9a94-d8df19369cfe None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:23.666902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 326/663] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:23.668442 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da3522b-f491-4377-9a94-d8df19369cfe None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:23.672609 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:23.702525 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c475fc9-b6eb-479d-a8ff-d0883b67b4c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:23.704644 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c475fc9-b6eb-479d-a8ff-d0883b67b4c3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:23.778644 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da3522b-f491-4377-9a94-d8df19369cfe None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:23.784592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:23.809400 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 338/664] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:18:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:18:23.810243 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c475fc9-b6eb-479d-a8ff-d0883b67b4c3 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:23.819367 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd2118-b57d-4898-b4ab-31364385e282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:23.819935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd2118-b57d-4898-b4ab-31364385e282 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:23.820716 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd2118-b57d-4898-b4ab-31364385e282 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:23.820997 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd2118-b57d-4898-b4ab-31364385e282 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:23.890253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd2118-b57d-4898-b4ab-31364385e282 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:23.891695 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd2118-b57d-4898-b4ab-31364385e282 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:23.894425 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:23.922517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c475fc9-b6eb-479d-a8ff-d0883b67b4c3 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:23.959445 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72df55e7-bc23-403d-9d4a-cb10f84b63d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:23.960034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df55e7-bc23-403d-9d4a-cb10f84b63d3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:23.960312 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df55e7-bc23-403d-9d4a-cb10f84b63d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:23.960584 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df55e7-bc23-403d-9d4a-cb10f84b63d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:23.961346 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72df55e7-bc23-403d-9d4a-cb10f84b63d3 None admin] RBAC: Adding query filter params (role.id=b0923c7a240149ae9ece5b3c1622e860, scope.project.id=479530ced580426b85d1fc07af43417f, user.id=6a2c872129414105a112661cc2b477e6) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:23.961650 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72df55e7-bc23-403d-9d4a-cb10f84b63d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:23.963103 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72df55e7-bc23-403d-9d4a-cb10f84b63d3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:23.969010 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 339/665] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:18:23 2022] GET /identity/v3/role_assignments?user.id=6a2c872129414105a112661cc2b477e6&scope.project.id=479530ced580426b85d1fc07af43417f&role.id=b0923c7a240149ae9ece5b3c1622e860 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:23.970180 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6471db-f1a1-4a9f-9027-b3875221ae5b None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:23.970649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6471db-f1a1-4a9f-9027-b3875221ae5b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:23.970815 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6471db-f1a1-4a9f-9027-b3875221ae5b None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:23.972157 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 340/666] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:23.972668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6471db-f1a1-4a9f-9027-b3875221ae5b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:24.016606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 327/667] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:18:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:18:24.017761 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1bff5-9cea-4829-a6b3-4d4b11a9c9a2 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:24.018045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1bff5-9cea-4829-a6b3-4d4b11a9c9a2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:24.018358 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1bff5-9cea-4829-a6b3-4d4b11a9c9a2 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:24.020164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 328/668] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:24.021618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493660c2-54c9-4a16-94b7-2445696b183b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:24.021618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493660c2-54c9-4a16-94b7-2445696b183b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:24.021848 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493660c2-54c9-4a16-94b7-2445696b183b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:24.040813 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-493660c2-54c9-4a16-94b7-2445696b183b None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:24.042349 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-493660c2-54c9-4a16-94b7-2445696b183b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:24.052149 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:24.121694 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6471db-f1a1-4a9f-9027-b3875221ae5b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:24.122220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 329/669] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:18:24.123080 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493660c2-54c9-4a16-94b7-2445696b183b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:24.164146 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:24.167396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aaff75-c10e-4a13-99d9-bdd806c52f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:24.167557 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aaff75-c10e-4a13-99d9-bdd806c52f58 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:24.167749 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aaff75-c10e-4a13-99d9-bdd806c52f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:24.167985 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aaff75-c10e-4a13-99d9-bdd806c52f58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:24.183414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aaff75-c10e-4a13-99d9-bdd806c52f58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:24.184926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aaff75-c10e-4a13-99d9-bdd806c52f58 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:24.203979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-98aaff75-c10e-4a13-99d9-bdd806c52f58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:24.203979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:24.203979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:24.203979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:24.203979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:24.203979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:24.203979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:24.203979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:24.203979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:24.203979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:24.203979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:24.203979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:24.203979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:24.203979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:24.203979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:24.203979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:24.205207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:24.205207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:24.205207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:24.205207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:24.205207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:24.205207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:24.205207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:24.205207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:24.205207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:24.205207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:24.205207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:24.205207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:24.205207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:24.205207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:24.205207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:24.205207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:24.205207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:24.206576 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:24.206576 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:24.206576 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:24.209757 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 341/670] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:24.211117 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d2cda-56d9-4da6-92f3-eacafb84d703 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:24.211406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d2cda-56d9-4da6-92f3-eacafb84d703 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:24.211683 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d2cda-56d9-4da6-92f3-eacafb84d703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:24.231588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e27d2cda-56d9-4da6-92f3-eacafb84d703 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:24.233119 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e27d2cda-56d9-4da6-92f3-eacafb84d703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:24.260997 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493660c2-54c9-4a16-94b7-2445696b183b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:24.300445 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba647993-f519-4e4f-9bcc-346aa3f86f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:24.301323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba647993-f519-4e4f-9bcc-346aa3f86f21 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:24.301781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba647993-f519-4e4f-9bcc-346aa3f86f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:24.302223 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba647993-f519-4e4f-9bcc-346aa3f86f21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:24.304385 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 342/671] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:18:24.305719 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d2cda-56d9-4da6-92f3-eacafb84d703 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:24.323474 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba647993-f519-4e4f-9bcc-346aa3f86f21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:24.325403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba647993-f519-4e4f-9bcc-346aa3f86f21 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:24.330987 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:24.347273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-ba647993-f519-4e4f-9bcc-346aa3f86f21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:24.347273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:24.347273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:24.347273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:24.347273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:24.347273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:24.347273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:24.347273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:24.347273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:24.347273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:24.347273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:24.347273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:24.347273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:24.347273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:24.347273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:24.347273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:24.348451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:24.348451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:24.348451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:24.348451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:24.348451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:24.348451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:24.348451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:24.348451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:24.348451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:24.348451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:24.348451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:24.348451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:24.348451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:24.348451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:24.348451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:24.348451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:24.348451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:24.349994 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:24.349994 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:24.349994 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:24.353120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 330/672] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:24.358901 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba647993-f519-4e4f-9bcc-346aa3f86f21 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:24.413320 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d2cda-56d9-4da6-92f3-eacafb84d703 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:24.449082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2311a5f-986c-40fe-8e3a-5efd201ac0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:24.449691 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2311a5f-986c-40fe-8e3a-5efd201ac0b3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:24.449971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2311a5f-986c-40fe-8e3a-5efd201ac0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:24.450315 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2311a5f-986c-40fe-8e3a-5efd201ac0b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:24.450951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2311a5f-986c-40fe-8e3a-5efd201ac0b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:24.451250 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2311a5f-986c-40fe-8e3a-5efd201ac0b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:24.452604 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2311a5f-986c-40fe-8e3a-5efd201ac0b3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:24.454711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:24.468995 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 343/673] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:24.471007 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2311a5f-986c-40fe-8e3a-5efd201ac0b3 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:24.478701 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba647993-f519-4e4f-9bcc-346aa3f86f21 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:24.521612 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c16f97-ea40-4724-9046-9536a7914610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:24.522646 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c16f97-ea40-4724-9046-9536a7914610 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:24.522931 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c16f97-ea40-4724-9046-9536a7914610 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:24.523206 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c16f97-ea40-4724-9046-9536a7914610 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:24.524409 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c16f97-ea40-4724-9046-9536a7914610 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:24.524778 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c16f97-ea40-4724-9046-9536a7914610 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:24.526634 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c16f97-ea40-4724-9046-9536a7914610 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:24.541819 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 331/674] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:24.546794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c16f97-ea40-4724-9046-9536a7914610 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:24.582174 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:24.600870 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2311a5f-986c-40fe-8e3a-5efd201ac0b3 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:24.648188 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1d313-9dca-4d93-9794-6a8f48ffbc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:24.649265 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1d313-9dca-4d93-9794-6a8f48ffbc26 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:24.649816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1d313-9dca-4d93-9794-6a8f48ffbc26 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:24.650314 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1d313-9dca-4d93-9794-6a8f48ffbc26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:24.662248 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1d313-9dca-4d93-9794-6a8f48ffbc26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:24.663798 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1d313-9dca-4d93-9794-6a8f48ffbc26 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:24.669595 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c16f97-ea40-4724-9046-9536a7914610 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:24.675662 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-3fe1d313-9dca-4d93-9794-6a8f48ffbc26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:24.675662 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:24.675662 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:24.675662 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:24.675662 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:24.675662 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:24.675662 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:24.675662 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:24.675662 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:24.675662 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:24.675662 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:24.675662 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:24.675662 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:24.675662 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:24.675662 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:24.676588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:24.676588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:24.676588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:24.676588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:24.676588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:24.676588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:24.676588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:24.676588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:24.676588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:24.676588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:24.676588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:24.676588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:24.676588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:24.676588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:24.676588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:24.676588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:24.676588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:24.678331 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:24.678331 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:24.678331 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:24.678331 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:24.678331 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:24.678331 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:24.678331 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:24.678331 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:24.681836 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 344/675] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:24.682753 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1d313-9dca-4d93-9794-6a8f48ffbc26 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:24.715344 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af8d9a-8655-43b1-b7e0-43c92d5a14fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:24.716262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af8d9a-8655-43b1-b7e0-43c92d5a14fe None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:24.716731 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af8d9a-8655-43b1-b7e0-43c92d5a14fe None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:24.717210 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af8d9a-8655-43b1-b7e0-43c92d5a14fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:24.728469 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af8d9a-8655-43b1-b7e0-43c92d5a14fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:24.734069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-28af8d9a-8655-43b1-b7e0-43c92d5a14fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:24.734069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:24.734069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:24.734069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:24.734069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:24.734069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:24.734069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:24.734069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:24.734069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:24.734069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:24.734069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:24.734069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:24.734069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:24.734069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:24.734069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:24.734069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:24.735057 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:24.735057 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:24.735057 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:24.735057 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:24.735057 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:24.735057 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:24.735057 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:24.735057 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:24.735057 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:24.735057 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:24.735057 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:24.735057 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:24.735057 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:24.735057 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:24.735057 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:24.735057 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:24.735925 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:24.735925 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:24.735925 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:24.735925 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:24.735925 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:24.735925 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:24.735925 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:24.735925 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:24.735925 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:24.735925 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:24.735925 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:24.735925 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:24.738823 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 332/676] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Sep 1 16:18:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:24.738823 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af8d9a-8655-43b1-b7e0-43c92d5a14fe None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:24.748586 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:24.789879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1d313-9dca-4d93-9794-6a8f48ffbc26 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:24.825440 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2e3c2-051e-42ca-b3f4-19acfbae4988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:24.826057 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2e3c2-051e-42ca-b3f4-19acfbae4988 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:24.826390 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2e3c2-051e-42ca-b3f4-19acfbae4988 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:24.826640 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2e3c2-051e-42ca-b3f4-19acfbae4988 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:24.833568 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2e3c2-051e-42ca-b3f4-19acfbae4988 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:24.835013 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2e3c2-051e-42ca-b3f4-19acfbae4988 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:24.845934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-7ad2e3c2-051e-42ca-b3f4-19acfbae4988 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:24.845934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:24.845934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:24.845934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:24.845934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:24.845934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:24.845934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:24.845934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:24.845934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:24.845934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:24.845934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:24.845934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:24.845934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:24.845934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:24.845934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:24.846898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:24.846898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:24.846898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:24.846898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:24.846898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:24.846898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:24.846898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:24.846898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:24.846898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:24.846898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:24.846898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:24.846898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:24.846898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:24.846898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:24.846898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:24.846898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:24.846898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:24.847820 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:24.847820 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:24.847820 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:24.847820 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:24.847820 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:24.847820 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:24.847820 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:24.847820 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:24.847820 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 345/677] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:24.847820 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2e3c2-051e-42ca-b3f4-19acfbae4988 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:24.854704 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:24.855503 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af8d9a-8655-43b1-b7e0-43c92d5a14fe None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:24.897885 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f6ce5-e74a-4417-a1d5-0109d3bc6cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:24.898784 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f6ce5-e74a-4417-a1d5-0109d3bc6cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:24.899224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f6ce5-e74a-4417-a1d5-0109d3bc6cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:24.899638 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f6ce5-e74a-4417-a1d5-0109d3bc6cd2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:24.911777 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f6ce5-e74a-4417-a1d5-0109d3bc6cd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:24.918123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-eb9f6ce5-e74a-4417-a1d5-0109d3bc6cd2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:18:24.918123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:24.918123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:24.918123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:24.918123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:24.918123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:24.918123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:24.918123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:24.918123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:24.918123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:24.918123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:24.918123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:24.918123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:24.918123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:24.918123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:24.918123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:24.919139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:24.919139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:24.919139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:24.919139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:24.919139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:24.919139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:24.919139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:24.919139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:24.919139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:24.919139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:24.919139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:24.919139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:24.919139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:24.919139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:24.919139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:24.919139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:24.920000 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:24.920000 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:24.920000 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:24.920000 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:24.920000 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:24.920000 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:24.920000 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:24.920000 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:24.920000 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:24.920000 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:24.920000 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:18:24.920000 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:24.920000 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 333/678] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:18:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:24.921579 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f6ce5-e74a-4417-a1d5-0109d3bc6cd2 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:24.958794 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2e3c2-051e-42ca-b3f4-19acfbae4988 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:24.998498 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:24.998851 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b358950-69d4-44df-b9ff-2aadf4e6f945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:24.999468 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b358950-69d4-44df-b9ff-2aadf4e6f945 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:24.999750 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b358950-69d4-44df-b9ff-2aadf4e6f945 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:25.000013 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b358950-69d4-44df-b9ff-2aadf4e6f945 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:25.000715 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b358950-69d4-44df-b9ff-2aadf4e6f945 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:25.001018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b358950-69d4-44df-b9ff-2aadf4e6f945 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:25.002470 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b358950-69d4-44df-b9ff-2aadf4e6f945 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:25.015224 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 346/679] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:25.016044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b358950-69d4-44df-b9ff-2aadf4e6f945 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:25.049691 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f6ce5-e74a-4417-a1d5-0109d3bc6cd2 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:25.092436 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf5765-4749-4a38-b1f5-c75be211cac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:25.093045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf5765-4749-4a38-b1f5-c75be211cac1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:25.093324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf5765-4749-4a38-b1f5-c75be211cac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:25.093598 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf5765-4749-4a38-b1f5-c75be211cac1 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:25.094297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf5765-4749-4a38-b1f5-c75be211cac1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:25.094603 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf5765-4749-4a38-b1f5-c75be211cac1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:25.095963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf5765-4749-4a38-b1f5-c75be211cac1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:25.099646 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:25.109857 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 334/680] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Sep 1 16:18:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:25.109857 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf5765-4749-4a38-b1f5-c75be211cac1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:25.148083 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b358950-69d4-44df-b9ff-2aadf4e6f945 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:25.187725 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c800a8-9d7f-4cd2-a3bd-c964ad6b482d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:25.188760 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c800a8-9d7f-4cd2-a3bd-c964ad6b482d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:25.189040 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c800a8-9d7f-4cd2-a3bd-c964ad6b482d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:25.189313 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c800a8-9d7f-4cd2-a3bd-c964ad6b482d None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:25.190016 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c800a8-9d7f-4cd2-a3bd-c964ad6b482d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:25.190804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c800a8-9d7f-4cd2-a3bd-c964ad6b482d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:25.192951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c800a8-9d7f-4cd2-a3bd-c964ad6b482d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:25.214018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 347/681] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:25.214939 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c800a8-9d7f-4cd2-a3bd-c964ad6b482d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:25.215420 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf5765-4749-4a38-b1f5-c75be211cac1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:25.245751 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:25.254843 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf92c7-80cb-41ce-84f3-90e9c59fdb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:25.255450 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf92c7-80cb-41ce-84f3-90e9c59fdb7b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:25.255733 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf92c7-80cb-41ce-84f3-90e9c59fdb7b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:25.256013 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf92c7-80cb-41ce-84f3-90e9c59fdb7b None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:25.256682 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf92c7-80cb-41ce-84f3-90e9c59fdb7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:25.256990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf92c7-80cb-41ce-84f3-90e9c59fdb7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:25.258496 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf92c7-80cb-41ce-84f3-90e9c59fdb7b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:25.271223 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 335/682] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:18:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:25.272122 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf92c7-80cb-41ce-84f3-90e9c59fdb7b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:25.338451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c800a8-9d7f-4cd2-a3bd-c964ad6b482d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:25.363986 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:25.379040 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ba514-2cde-4cdf-9c83-3193a64bd6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:25.379974 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ba514-2cde-4cdf-9c83-3193a64bd6f3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:25.380397 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ba514-2cde-4cdf-9c83-3193a64bd6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:25.381090 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ba514-2cde-4cdf-9c83-3193a64bd6f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:25.382119 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1ba514-2cde-4cdf-9c83-3193a64bd6f3 None admin] RBAC: Adding query filter params (role.id=b0923c7a240149ae9ece5b3c1622e860, scope.project.id=479530ced580426b85d1fc07af43417f, user.id=84d4bd0a456341f78b5e476c97a9165c) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:25.382680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1ba514-2cde-4cdf-9c83-3193a64bd6f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:25.386027 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1ba514-2cde-4cdf-9c83-3193a64bd6f3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:25.396189 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 348/683] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:18:25 2022] GET /identity/v3/role_assignments?user.id=84d4bd0a456341f78b5e476c97a9165c&scope.project.id=479530ced580426b85d1fc07af43417f&role.id=b0923c7a240149ae9ece5b3c1622e860 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:25.397226 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ba514-2cde-4cdf-9c83-3193a64bd6f3 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:25.419541 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf92c7-80cb-41ce-84f3-90e9c59fdb7b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:25.459129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440db7e0-02b9-4523-8da7-423f5bf4f1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:25.460200 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440db7e0-02b9-4523-8da7-423f5bf4f1b0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:25.460680 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440db7e0-02b9-4523-8da7-423f5bf4f1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:25.461138 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440db7e0-02b9-4523-8da7-423f5bf4f1b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:25.475278 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440db7e0-02b9-4523-8da7-423f5bf4f1b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:25.478250 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440db7e0-02b9-4523-8da7-423f5bf4f1b0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:25.497639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-440db7e0-02b9-4523-8da7-423f5bf4f1b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:25.497639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:25.497639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:25.497639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:25.497639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:25.497639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:25.497639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:25.497639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:25.497639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:25.497639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:25.497639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:25.497639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:25.497639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:25.497639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:25.497639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:25.497639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:25.498724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:25.498724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:25.498724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:25.498724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:25.498724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:25.498724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:25.498724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:25.498724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:25.498724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:25.498724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:25.498724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:25.498724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:25.498724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:25.498724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:25.498724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:25.498724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:25.498724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:25.499658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:25.499658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:25.499658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:25.506604 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 336/684] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:25.508567 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440db7e0-02b9-4523-8da7-423f5bf4f1b0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:25.538365 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ba514-2cde-4cdf-9c83-3193a64bd6f3 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:25.564757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:25.588317 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4a368-0b0d-4c1f-a655-c5b9e871e151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:25.588879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4a368-0b0d-4c1f-a655-c5b9e871e151 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:25.589631 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4a368-0b0d-4c1f-a655-c5b9e871e151 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:25.589631 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4a368-0b0d-4c1f-a655-c5b9e871e151 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:25.590150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4a368-0b0d-4c1f-a655-c5b9e871e151 None admin] RBAC: Adding query filter params (role.id=b0923c7a240149ae9ece5b3c1622e860, scope.project.id=479530ced580426b85d1fc07af43417f, user.id=943fae29c5dc41f2911bbe051c74a361) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:25.590547 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4a368-0b0d-4c1f-a655-c5b9e871e151 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:25.591981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4a368-0b0d-4c1f-a655-c5b9e871e151 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:25.603285 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 349/685] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:18:25 2022] GET /identity/v3/role_assignments?user.id=943fae29c5dc41f2911bbe051c74a361&scope.project.id=479530ced580426b85d1fc07af43417f&role.id=b0923c7a240149ae9ece5b3c1622e860 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:25.604965 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4a368-0b0d-4c1f-a655-c5b9e871e151 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:25.648123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440db7e0-02b9-4523-8da7-423f5bf4f1b0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:25.688240 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:25.693809 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2724b8b-db6a-4793-8738-6868a98d625a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:25.694353 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2724b8b-db6a-4793-8738-6868a98d625a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:25.694641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2724b8b-db6a-4793-8738-6868a98d625a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:25.695003 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2724b8b-db6a-4793-8738-6868a98d625a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:25.727292 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2724b8b-db6a-4793-8738-6868a98d625a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:25.728913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2724b8b-db6a-4793-8738-6868a98d625a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:25.753787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4a368-0b0d-4c1f-a655-c5b9e871e151 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:25.763276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-b2724b8b-db6a-4793-8738-6868a98d625a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:25.763276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:25.763276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:25.763276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:25.763276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:25.763276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:25.763276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:25.763276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:25.763276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:25.763276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:25.763276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:25.763276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:25.763276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:25.763276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:25.763276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:25.763276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:25.764492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:25.764492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:25.764492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:25.764492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:25.764492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:25.764492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:25.764492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:25.764492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:25.764492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:25.764492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:25.764492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:25.764492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:25.764492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:25.764492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:25.764492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:25.764492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:25.764492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:25.770133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:25.770133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:25.770133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:25.770133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 337/686] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:25.781400 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2724b8b-db6a-4793-8738-6868a98d625a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:25.800201 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce226f-64d5-420f-b146-8aed17670d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:25.801394 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce226f-64d5-420f-b146-8aed17670d18 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:25.801817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce226f-64d5-420f-b146-8aed17670d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:25.802262 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce226f-64d5-420f-b146-8aed17670d18 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:25.803054 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce226f-64d5-420f-b146-8aed17670d18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:25.803518 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce226f-64d5-420f-b146-8aed17670d18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:25.805553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce226f-64d5-420f-b146-8aed17670d18 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:25.820702 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 350/687] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:25.826994 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce226f-64d5-420f-b146-8aed17670d18 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:25.937568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:25.948143 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2724b8b-db6a-4793-8738-6868a98d625a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:25.960472 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce226f-64d5-420f-b146-8aed17670d18 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:25.989893 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:26.012254 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee927ad-aa3a-4136-a12b-67ff9c7db0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:26.013105 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee927ad-aa3a-4136-a12b-67ff9c7db0a8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:26.013544 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee927ad-aa3a-4136-a12b-67ff9c7db0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:26.014136 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee927ad-aa3a-4136-a12b-67ff9c7db0a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:26.015959 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee927ad-aa3a-4136-a12b-67ff9c7db0a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:26.017171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee927ad-aa3a-4136-a12b-67ff9c7db0a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:26.019315 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee927ad-aa3a-4136-a12b-67ff9c7db0a8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:26.035374 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca201db-a201-4d0f-b3e3-10589521613b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:26.036281 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca201db-a201-4d0f-b3e3-10589521613b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:26.036706 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca201db-a201-4d0f-b3e3-10589521613b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:26.037131 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca201db-a201-4d0f-b3e3-10589521613b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:26.042918 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 338/688] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:26.051961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee927ad-aa3a-4136-a12b-67ff9c7db0a8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:26.053770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca201db-a201-4d0f-b3e3-10589521613b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:26.055901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca201db-a201-4d0f-b3e3-10589521613b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:26.071280 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-2ca201db-a201-4d0f-b3e3-10589521613b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:18:26.071280 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:26.071280 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:26.071280 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:26.071280 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:26.071280 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:26.071280 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:26.071280 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:26.071280 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:26.071280 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:26.071280 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:26.071280 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:26.071280 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:26.071280 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:26.071280 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:26.071280 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:26.072354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:26.072354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:26.072354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:26.072354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:26.072354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:26.072354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:26.072354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:26.072354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:26.072354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:26.072354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:26.072354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:26.072354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:26.072354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:26.072354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:26.072354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:26.072354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:26.072354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:26.072354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:26.073321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:26.073321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:26.073321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:26.073321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:26.073321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:26.073321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:26.073321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:26.073321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:26.073321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:18:26.073321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:26.073321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 351/689] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:18:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:26.073940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85f008-2e6c-4e6b-93cf-2f7e06333375 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:26.074554 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85f008-2e6c-4e6b-93cf-2f7e06333375 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:26.078726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85f008-2e6c-4e6b-93cf-2f7e06333375 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:26.080148 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 352/690] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:26.082251 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85f008-2e6c-4e6b-93cf-2f7e06333375 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:26.173957 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee927ad-aa3a-4136-a12b-67ff9c7db0a8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:26.196668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85f008-2e6c-4e6b-93cf-2f7e06333375 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:26.209220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02f63c-435a-4cae-b632-1b9822eab2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:26.210277 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02f63c-435a-4cae-b632-1b9822eab2e0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:26.211293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02f63c-435a-4cae-b632-1b9822eab2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:26.211817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02f63c-435a-4cae-b632-1b9822eab2e0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:26.221373 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba02f63c-435a-4cae-b632-1b9822eab2e0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:26.223630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba02f63c-435a-4cae-b632-1b9822eab2e0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:26.234398 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb8a6a-b1ae-45de-8041-a85ba475909f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:26.235015 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb8a6a-b1ae-45de-8041-a85ba475909f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:26.235357 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb8a6a-b1ae-45de-8041-a85ba475909f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:26.235638 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb8a6a-b1ae-45de-8041-a85ba475909f None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:26.236387 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb8a6a-b1ae-45de-8041-a85ba475909f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:26.236695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb8a6a-b1ae-45de-8041-a85ba475909f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:26.238123 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb8a6a-b1ae-45de-8041-a85ba475909f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:26.240228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-ba02f63c-435a-4cae-b632-1b9822eab2e0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:18:26.240228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:26.240228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:26.240228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:26.240228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:26.240228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:26.240228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:26.240228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:26.240228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:26.240228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:26.240228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:26.240228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:26.240228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:26.240228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:26.240228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:26.240228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:26.241150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:26.241150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:26.241150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:26.241150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:26.241150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:26.241150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:26.241150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:26.241150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:26.241150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:26.241150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:26.241150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:26.241150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:26.241150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:26.241150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:26.241150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:26.241150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:26.241150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:26.241150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:26.242161 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:26.242161 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:26.242161 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:26.242161 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:26.242161 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:26.242161 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:26.242161 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:26.242161 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:26.242161 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:18:26.242161 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:26.246269 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 339/691] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Sep 1 16:18:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:26.247789 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a352308-f672-4996-9b6e-ea72c2272ebc None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:26.248274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a352308-f672-4996-9b6e-ea72c2272ebc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:26.248764 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a352308-f672-4996-9b6e-ea72c2272ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:26.268931 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 353/692] 10.222.0.58 () {56 vars in 1135 bytes} [Thu Sep 1 16:18:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:18:26.269889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb8a6a-b1ae-45de-8041-a85ba475909f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:26.270414 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9a352308-f672-4996-9b6e-ea72c2272ebc None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:26.271874 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9a352308-f672-4996-9b6e-ea72c2272ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:26.285646 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:26.334672 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:26.366600 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 340/693] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:18:26.366600 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a352308-f672-4996-9b6e-ea72c2272ebc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:26.396288 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb8a6a-b1ae-45de-8041-a85ba475909f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:26.464773 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd830b9-b452-4c5c-aae3-dc0934784501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:26.465387 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd830b9-b452-4c5c-aae3-dc0934784501 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:26.465669 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd830b9-b452-4c5c-aae3-dc0934784501 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:26.465944 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd830b9-b452-4c5c-aae3-dc0934784501 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:26.466709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd830b9-b452-4c5c-aae3-dc0934784501 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:26.467269 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd830b9-b452-4c5c-aae3-dc0934784501 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:26.469553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd830b9-b452-4c5c-aae3-dc0934784501 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:26.509113 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 354/694] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Sep 1 16:18:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:26.514450 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd830b9-b452-4c5c-aae3-dc0934784501 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:26.550400 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a352308-f672-4996-9b6e-ea72c2272ebc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:26.607026 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da6273-4969-4b9d-b7f0-ce0800151ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:26.607647 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da6273-4969-4b9d-b7f0-ce0800151ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:26.608211 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da6273-4969-4b9d-b7f0-ce0800151ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:26.608493 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da6273-4969-4b9d-b7f0-ce0800151ca8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:26.623580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:26.688278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:26.690933 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25da6273-4969-4b9d-b7f0-ce0800151ca8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:26.692411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25da6273-4969-4b9d-b7f0-ce0800151ca8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:26.701613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd830b9-b452-4c5c-aae3-dc0934784501 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:26.738967 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec466a6-436c-4116-b99a-ec9d681ee28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:26.740056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec466a6-436c-4116-b99a-ec9d681ee28a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:26.740571 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec466a6-436c-4116-b99a-ec9d681ee28a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:26.741042 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec466a6-436c-4116-b99a-ec9d681ee28a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:26.804241 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 341/695] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:18:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:18:26.805594 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da6273-4969-4b9d-b7f0-ce0800151ca8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:26.818029 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec466a6-436c-4116-b99a-ec9d681ee28a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:26.820151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec466a6-436c-4116-b99a-ec9d681ee28a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:26.855149 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:26.915162 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da6273-4969-4b9d-b7f0-ce0800151ca8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:26.915902 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:26.930346 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 355/696] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:18:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:18:26.931499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec466a6-436c-4116-b99a-ec9d681ee28a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:26.955756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d2cd0-91df-42a1-9cae-f57beca82184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:26.957046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d2cd0-91df-42a1-9cae-f57beca82184 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:26.957485 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d2cd0-91df-42a1-9cae-f57beca82184 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:26.958015 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d2cd0-91df-42a1-9cae-f57beca82184 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:26.970925 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d2cd0-91df-42a1-9cae-f57beca82184 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:26.972754 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d2cd0-91df-42a1-9cae-f57beca82184 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:26.989805 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-2d2d2cd0-91df-42a1-9cae-f57beca82184 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:26.989805 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:26.989805 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:26.989805 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:26.989805 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:26.989805 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:26.989805 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:26.989805 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:26.989805 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:26.989805 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:26.989805 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:26.989805 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:26.989805 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:26.989805 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:26.989805 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:26.989805 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:26.990880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:26.990880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:26.990880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:26.990880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:26.990880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:26.990880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:26.990880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:26.990880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:26.990880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:26.990880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:26.990880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:26.990880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:26.990880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:26.990880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:26.990880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:26.990880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:26.990880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:26.991841 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:26.991841 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:26.991841 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:26.994277 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 342/697] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:26.995201 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d2cd0-91df-42a1-9cae-f57beca82184 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:27.049527 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec466a6-436c-4116-b99a-ec9d681ee28a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:27.100647 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca7b7b-5d63-4ec9-8cd8-164fc9f533c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:27.101438 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca7b7b-5d63-4ec9-8cd8-164fc9f533c2 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:27.101866 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca7b7b-5d63-4ec9-8cd8-164fc9f533c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:27.102427 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca7b7b-5d63-4ec9-8cd8-164fc9f533c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:27.114228 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca7b7b-5d63-4ec9-8cd8-164fc9f533c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:27.115858 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca7b7b-5d63-4ec9-8cd8-164fc9f533c2 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:27.118307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:27.125552 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d2cd0-91df-42a1-9cae-f57beca82184 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:27.134739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-26ca7b7b-5d63-4ec9-8cd8-164fc9f533c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:27.134739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:27.134739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:27.134739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:27.134739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:27.134739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:27.134739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:27.134739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:27.134739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:27.134739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:27.134739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:27.134739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:27.134739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:27.134739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:27.134739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:27.134739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:27.135872 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:27.135872 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:27.135872 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:27.135872 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:27.135872 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:27.135872 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:27.135872 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:27.135872 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:27.135872 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:27.135872 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:27.135872 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:27.135872 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:27.135872 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:27.135872 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:27.135872 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:27.135872 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:27.135872 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:27.136836 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:27.136836 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:27.136836 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:27.136836 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 356/698] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:27.137763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca7b7b-5d63-4ec9-8cd8-164fc9f533c2 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:27.167967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db1f42-5482-4993-aee4-4488f2d0b5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:27.169299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db1f42-5482-4993-aee4-4488f2d0b5db None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:27.169789 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db1f42-5482-4993-aee4-4488f2d0b5db None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:27.170295 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db1f42-5482-4993-aee4-4488f2d0b5db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:27.186527 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db1f42-5482-4993-aee4-4488f2d0b5db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:27.189422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db1f42-5482-4993-aee4-4488f2d0b5db None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:27.196468 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:27.211168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-c1db1f42-5482-4993-aee4-4488f2d0b5db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:27.211168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:27.211168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:27.211168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:27.211168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:27.211168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:27.211168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:27.211168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:27.211168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:27.211168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:27.211168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:27.211168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:27.211168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:27.211168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:27.211168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:27.211168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:27.212419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:27.212419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:27.212419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:27.212419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:27.212419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:27.212419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:27.212419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:27.212419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:27.212419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:27.212419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:27.212419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:27.212419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:27.212419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:27.212419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:27.212419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:27.212419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:27.212419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:27.213561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:27.213561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:27.213561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:27.216488 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 343/699] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:27.217565 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db1f42-5482-4993-aee4-4488f2d0b5db None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:27.268635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca7b7b-5d63-4ec9-8cd8-164fc9f533c2 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:27.305607 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88626fd1-b3e4-43c4-92cf-a32319474dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:27.306582 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88626fd1-b3e4-43c4-92cf-a32319474dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:27.307174 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88626fd1-b3e4-43c4-92cf-a32319474dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:27.307631 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88626fd1-b3e4-43c4-92cf-a32319474dc7 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:27.308508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88626fd1-b3e4-43c4-92cf-a32319474dc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:27.308994 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88626fd1-b3e4-43c4-92cf-a32319474dc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:27.310576 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88626fd1-b3e4-43c4-92cf-a32319474dc7 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:27.325360 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 357/700] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:27.326496 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88626fd1-b3e4-43c4-92cf-a32319474dc7 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:27.328065 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db1f42-5482-4993-aee4-4488f2d0b5db None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:27.364635 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3383106a-b77a-450a-99d4-ba1255f24d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:27.365251 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383106a-b77a-450a-99d4-ba1255f24d7e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:27.365603 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383106a-b77a-450a-99d4-ba1255f24d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:27.365950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383106a-b77a-450a-99d4-ba1255f24d7e None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:27.366718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3383106a-b77a-450a-99d4-ba1255f24d7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:27.367127 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3383106a-b77a-450a-99d4-ba1255f24d7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:27.368595 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3383106a-b77a-450a-99d4-ba1255f24d7e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:27.385977 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 344/701] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:27.387007 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3383106a-b77a-450a-99d4-ba1255f24d7e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:27.397975 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:27.434860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:27.438362 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88626fd1-b3e4-43c4-92cf-a32319474dc7 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:27.475770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33277222-3079-4ab1-a3c4-7638713efca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:27.476989 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33277222-3079-4ab1-a3c4-7638713efca1 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:27.477469 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33277222-3079-4ab1-a3c4-7638713efca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:27.477975 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33277222-3079-4ab1-a3c4-7638713efca1 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:27.478890 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33277222-3079-4ab1-a3c4-7638713efca1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:27.479405 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33277222-3079-4ab1-a3c4-7638713efca1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:27.481104 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33277222-3079-4ab1-a3c4-7638713efca1 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:27.494996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 358/702] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:27.496208 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33277222-3079-4ab1-a3c4-7638713efca1 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:27.503477 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3383106a-b77a-450a-99d4-ba1255f24d7e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:27.543258 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4562b9-0f99-4d9e-b844-962aec473f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:27.543659 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4562b9-0f99-4d9e-b844-962aec473f0e None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:27.543947 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4562b9-0f99-4d9e-b844-962aec473f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:27.544224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4562b9-0f99-4d9e-b844-962aec473f0e None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:27.544872 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4562b9-0f99-4d9e-b844-962aec473f0e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:27.546460 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4562b9-0f99-4d9e-b844-962aec473f0e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:27.578220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 345/703] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Sep 1 16:18:27 2022] POST /identity/v3/users => generated 254 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:18:27.578220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4562b9-0f99-4d9e-b844-962aec473f0e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:27.634301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33277222-3079-4ab1-a3c4-7638713efca1 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:27.674096 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6a168-13f1-4e87-879e-29f4a94cded8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:27.674632 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6a168-13f1-4e87-879e-29f4a94cded8 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:27.674923 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6a168-13f1-4e87-879e-29f4a94cded8 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:27.675200 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6a168-13f1-4e87-879e-29f4a94cded8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:27.678969 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:27.684706 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b6a168-13f1-4e87-879e-29f4a94cded8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:27.685855 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b6a168-13f1-4e87-879e-29f4a94cded8 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:27.700638 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-61b6a168-13f1-4e87-879e-29f4a94cded8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:27.700638 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:27.700638 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:27.700638 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:27.700638 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:27.700638 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:27.700638 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:27.700638 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:27.700638 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:27.700638 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:27.700638 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:27.700638 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:27.700638 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:27.700638 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:27.700638 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:27.701635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:27.701635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:27.701635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:27.701635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:27.701635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:27.701635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:27.701635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:27.701635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:27.701635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:27.701635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:27.701635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:27.701635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:27.701635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:27.701635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:27.701635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:27.701635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:27.701635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:27.702853 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:27.702853 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:27.702853 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:27.702853 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:27.702853 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:27.702853 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:27.702853 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:27.702853 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:27.702853 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 359/704] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:27.709045 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf0bc4-3758-4b30-9c9e-773948ea703f None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:27.709393 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf0bc4-3758-4b30-9c9e-773948ea703f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:27.709742 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf0bc4-3758-4b30-9c9e-773948ea703f None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:27.712042 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 360/705] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:27 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:27.712937 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf0bc4-3758-4b30-9c9e-773948ea703f None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:27.732162 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4562b9-0f99-4d9e-b844-962aec473f0e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:27.738637 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:27.776931 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b044bb17-be42-417c-a58a-adf93067fefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:27.777712 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044bb17-be42-417c-a58a-adf93067fefc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:27.777896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044bb17-be42-417c-a58a-adf93067fefc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:27.778134 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044bb17-be42-417c-a58a-adf93067fefc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:27.788085 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b044bb17-be42-417c-a58a-adf93067fefc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:27.790148 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b044bb17-be42-417c-a58a-adf93067fefc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:27.804737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-b044bb17-be42-417c-a58a-adf93067fefc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:27.804737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:27.804737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:27.804737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:27.804737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:27.804737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:27.804737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:27.804737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:27.804737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:27.804737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:27.804737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:27.804737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:27.804737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:27.804737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:27.804737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:27.805704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:27.805704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:27.805704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:27.805704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:27.805704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:27.805704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:27.805704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:27.805704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:27.805704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:27.805704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:27.805704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:27.805704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:27.805704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:27.805704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:27.805704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:27.805704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:27.805704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:27.807002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:27.807002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:27.807002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:27.807002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:27.807002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:27.807002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:27.807002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:27.807002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:27.807002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 346/706] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:27.807965 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e52e76-c4b5-45fb-ada3-36ddb35f8af0 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:27.807965 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e52e76-c4b5-45fb-ada3-36ddb35f8af0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:27.808217 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e52e76-c4b5-45fb-ada3-36ddb35f8af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:27.828084 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-37e52e76-c4b5-45fb-ada3-36ddb35f8af0 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:27.829155 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-37e52e76-c4b5-45fb-ada3-36ddb35f8af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:27.853909 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf0bc4-3758-4b30-9c9e-773948ea703f None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:27.890325 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7267d783-049c-4a1c-ab96-00c19cf63d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:27.890857 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267d783-049c-4a1c-ab96-00c19cf63d21 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:27.891141 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267d783-049c-4a1c-ab96-00c19cf63d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:27.891471 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267d783-049c-4a1c-ab96-00c19cf63d21 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:27.892279 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7267d783-049c-4a1c-ab96-00c19cf63d21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:27.892649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7267d783-049c-4a1c-ab96-00c19cf63d21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:27.894010 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7267d783-049c-4a1c-ab96-00c19cf63d21 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:27.900026 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 347/707] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:18:27.901505 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d55ce-82f6-4c85-85bc-c5e82ebb0dfa None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:27.902503 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d55ce-82f6-4c85-85bc-c5e82ebb0dfa None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:27.903013 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d55ce-82f6-4c85-85bc-c5e82ebb0dfa None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:27.905475 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 348/708] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:27.906423 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d55ce-82f6-4c85-85bc-c5e82ebb0dfa None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:27.915948 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 361/709] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:27.915948 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f883c52-47b9-4155-b18e-859880fc5051 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:27.915948 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f883c52-47b9-4155-b18e-859880fc5051 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:27.915948 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f883c52-47b9-4155-b18e-859880fc5051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:27.931467 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2f883c52-47b9-4155-b18e-859880fc5051 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:27.932634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2f883c52-47b9-4155-b18e-859880fc5051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:27.984026 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:28.010229 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 362/710] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:18:28.010229 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f883c52-47b9-4155-b18e-859880fc5051 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:28.024025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:28.067759 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d55ce-82f6-4c85-85bc-c5e82ebb0dfa None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:28.139853 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfff9f-0f2f-430f-a2da-d7590f6f407b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:28.140527 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfff9f-0f2f-430f-a2da-d7590f6f407b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:28.140757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfff9f-0f2f-430f-a2da-d7590f6f407b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:28.141036 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfff9f-0f2f-430f-a2da-d7590f6f407b None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:28.141737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bfff9f-0f2f-430f-a2da-d7590f6f407b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:28.142044 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bfff9f-0f2f-430f-a2da-d7590f6f407b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:28.143503 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bfff9f-0f2f-430f-a2da-d7590f6f407b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:28.156471 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f883c52-47b9-4155-b18e-859880fc5051 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:28.173550 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 349/711] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:28.179053 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfff9f-0f2f-430f-a2da-d7590f6f407b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:28.197688 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ffaab-9c94-440d-9b35-81cde4f0a7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:28.198312 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ffaab-9c94-440d-9b35-81cde4f0a7c2 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:28.198592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ffaab-9c94-440d-9b35-81cde4f0a7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:28.198861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ffaab-9c94-440d-9b35-81cde4f0a7c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:28.199590 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90ffaab-9c94-440d-9b35-81cde4f0a7c2 None admin] RBAC: Adding query filter params (role.id=b0923c7a240149ae9ece5b3c1622e860, scope.project.id=479530ced580426b85d1fc07af43417f, user.id=e789695517c1437fb224f5a6195655f9) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:28.199893 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90ffaab-9c94-440d-9b35-81cde4f0a7c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:28.201321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90ffaab-9c94-440d-9b35-81cde4f0a7c2 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:28.212024 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 363/712] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:18:28 2022] GET /identity/v3/role_assignments?user.id=e789695517c1437fb224f5a6195655f9&scope.project.id=479530ced580426b85d1fc07af43417f&role.id=b0923c7a240149ae9ece5b3c1622e860 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:28.220947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ffaab-9c94-440d-9b35-81cde4f0a7c2 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:28.305937 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfff9f-0f2f-430f-a2da-d7590f6f407b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:28.322226 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:28.360947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:28.371571 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c5b16-dbb1-435f-b30a-00a21937218c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:28.372530 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c5b16-dbb1-435f-b30a-00a21937218c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:28.372820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c5b16-dbb1-435f-b30a-00a21937218c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:28.373101 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c5b16-dbb1-435f-b30a-00a21937218c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:28.374472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8c5b16-dbb1-435f-b30a-00a21937218c None admin] RBAC: Adding query filter params (role.id=ff293acb8551463aafe6970230042b89, scope.project.id=479530ced580426b85d1fc07af43417f, user.id=1f0c99fdf8bd4331ad7e641ee38554c1) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:28.374797 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8c5b16-dbb1-435f-b30a-00a21937218c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:28.376906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8c5b16-dbb1-435f-b30a-00a21937218c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:28.386034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 350/713] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:18:28 2022] GET /identity/v3/role_assignments?user.id=1f0c99fdf8bd4331ad7e641ee38554c1&scope.project.id=479530ced580426b85d1fc07af43417f&role.id=ff293acb8551463aafe6970230042b89 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:28.390435 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c5b16-dbb1-435f-b30a-00a21937218c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:28.405913 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ffaab-9c94-440d-9b35-81cde4f0a7c2 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:28.459607 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a44d2-d1c4-4cd0-894d-1022c50bb6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:28.460510 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a44d2-d1c4-4cd0-894d-1022c50bb6ba None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:28.461463 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a44d2-d1c4-4cd0-894d-1022c50bb6ba None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:28.461995 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a44d2-d1c4-4cd0-894d-1022c50bb6ba None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:28.462823 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a44d2-d1c4-4cd0-894d-1022c50bb6ba None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:28.465164 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a44d2-d1c4-4cd0-894d-1022c50bb6ba None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:28.477942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-8a1a44d2-d1c4-4cd0-894d-1022c50bb6ba None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:18:28.477942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:28.477942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:18:28.477942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:18:28.477942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:18:28.477942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:18:28.477942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:18:28.477942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:28.477942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:18:28.477942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:28.477942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:18:28.477942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:28.477942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:28.477942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:28.477942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:28.477942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:28.477942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:28.479187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:28.479187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:28.479187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:28.479187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:28.479187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:28.479187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:28.479187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:18:28.479187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:18:28.479187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:18:28.479187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:18:28.479187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:28.479187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:28.479187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:28.479187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:28.479187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:28.479187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:28.480070 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:28.480070 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:28.480070 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:28.480070 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:28.480070 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:28.480070 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:28.480070 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:28.480070 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:28.480070 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:28.480070 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:28.480070 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:18:28.480070 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:28.480070 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:18:28.480070 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:28.482977 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 364/714] 10.222.0.58 () {56 vars in 1129 bytes} [Thu Sep 1 16:18:28 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:18:28.488963 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a44d2-d1c4-4cd0-894d-1022c50bb6ba None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:28.529430 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c5b16-dbb1-435f-b30a-00a21937218c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:28.585261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dec237-4e3f-4f04-b368-a26d9cdd408f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:28.586313 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dec237-4e3f-4f04-b368-a26d9cdd408f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:28.586805 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dec237-4e3f-4f04-b368-a26d9cdd408f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:28.587282 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dec237-4e3f-4f04-b368-a26d9cdd408f None admin] PATH_INFO: `/v3/services/network` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:28.588105 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dec237-4e3f-4f04-b368-a26d9cdd408f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:28.590398 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dec237-4e3f-4f04-b368-a26d9cdd408f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:28.603457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-82dec237-4e3f-4f04-b368-a26d9cdd408f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:18:28.603457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:28.603457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:18:28.603457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:18:28.603457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:18:28.603457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:18:28.603457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:18:28.603457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:28.603457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:18:28.603457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:28.603457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:18:28.603457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:28.603457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:28.603457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:28.603457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:28.603457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:28.603457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:28.604528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:28.604528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:28.604528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:28.604528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:28.604528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:28.604528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:28.604528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:18:28.604528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:18:28.604528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:18:28.604528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:18:28.604528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:28.604528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:28.604528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:28.604528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:28.604528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:28.604528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:28.605399 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:28.605399 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:28.605399 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:28.605399 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:28.605399 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:28.605399 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:28.605399 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:28.605399 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:28.605399 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:28.605399 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:28.605399 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:18:28.605399 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:28.605399 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:18:28.605399 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:28.608243 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 351/715] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:28 2022] GET /identity/v3/services/network => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:28.614361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dec237-4e3f-4f04-b368-a26d9cdd408f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:28.619793 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a44d2-d1c4-4cd0-894d-1022c50bb6ba None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:28.659807 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6ab03-99b6-4e13-8e7b-2da2dd078b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:28.660800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6ab03-99b6-4e13-8e7b-2da2dd078b14 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:28.661283 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6ab03-99b6-4e13-8e7b-2da2dd078b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:28.661840 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6ab03-99b6-4e13-8e7b-2da2dd078b14 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:28.662720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef6ab03-99b6-4e13-8e7b-2da2dd078b14 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:28.663248 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef6ab03-99b6-4e13-8e7b-2da2dd078b14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:28.665034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef6ab03-99b6-4e13-8e7b-2da2dd078b14 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:28.666058 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:28.671621 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 365/716] 10.222.0.58 () {56 vars in 1124 bytes} [Thu Sep 1 16:18:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:28.678545 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6ab03-99b6-4e13-8e7b-2da2dd078b14 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:28.682124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:28.732023 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dec237-4e3f-4f04-b368-a26d9cdd408f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:28.768903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbddd769-5712-406c-9025-a11cecaa4338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:28.769683 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbddd769-5712-406c-9025-a11cecaa4338 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:28.770061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbddd769-5712-406c-9025-a11cecaa4338 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:28.770463 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbddd769-5712-406c-9025-a11cecaa4338 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:28.771218 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbddd769-5712-406c-9025-a11cecaa4338 None admin] RBAC: Adding query filter params (name=network) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:28.771596 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbddd769-5712-406c-9025-a11cecaa4338 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:28.773080 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbddd769-5712-406c-9025-a11cecaa4338 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:28.778245 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 352/717] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Sep 1 16:18:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:28.783912 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbddd769-5712-406c-9025-a11cecaa4338 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:28.797708 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6ab03-99b6-4e13-8e7b-2da2dd078b14 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:28.837963 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773ff98-6af5-4b01-a074-a8c0219e6855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:28.838842 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773ff98-6af5-4b01-a074-a8c0219e6855 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:28.839410 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773ff98-6af5-4b01-a074-a8c0219e6855 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:28.839889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773ff98-6af5-4b01-a074-a8c0219e6855 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:28.840800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773ff98-6af5-4b01-a074-a8c0219e6855 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:28.841247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773ff98-6af5-4b01-a074-a8c0219e6855 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:28.842926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773ff98-6af5-4b01-a074-a8c0219e6855 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:28.849895 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 366/718] 10.222.0.58 () {56 vars in 1124 bytes} [Thu Sep 1 16:18:28 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:28.892979 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbddd769-5712-406c-9025-a11cecaa4338 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:28.902154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:28.932288 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c2212-1eea-40d1-b9ee-a49637d145c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:28.932830 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c2212-1eea-40d1-b9ee-a49637d145c8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:28.933113 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c2212-1eea-40d1-b9ee-a49637d145c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:28.933388 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c2212-1eea-40d1-b9ee-a49637d145c8 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:28.934029 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c2212-1eea-40d1-b9ee-a49637d145c8 None admin] RBAC: Adding query filter params (type=network) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:28.934360 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c2212-1eea-40d1-b9ee-a49637d145c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:28.935738 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c2212-1eea-40d1-b9ee-a49637d145c8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:28.941992 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 353/719] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Sep 1 16:18:28 2022] GET /identity/v3/services?type=network => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:29.867979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585ee5d-0461-4113-9882-74c91c15bb14 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:29.868535 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585ee5d-0461-4113-9882-74c91c15bb14 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:29.868535 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585ee5d-0461-4113-9882-74c91c15bb14 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:29.870117 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 367/720] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:29.887431 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d0c6d-4b94-434b-8cdd-37ce1f4ec8f4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:29.888357 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d0c6d-4b94-434b-8cdd-37ce1f4ec8f4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:29.888878 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d0c6d-4b94-434b-8cdd-37ce1f4ec8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:29.911676 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1a2d0c6d-4b94-434b-8cdd-37ce1f4ec8f4 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:29.913214 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1a2d0c6d-4b94-434b-8cdd-37ce1f4ec8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:30.002795 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 354/721] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:18:30.339602 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585ee5d-0461-4113-9882-74c91c15bb14 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:30.422162 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:30.462711 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585ee5d-0461-4113-9882-74c91c15bb14 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:30.523046 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce592f68-3d1c-4c7b-b5dd-b9d6b573703f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:30.524124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce592f68-3d1c-4c7b-b5dd-b9d6b573703f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:30.524645 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce592f68-3d1c-4c7b-b5dd-b9d6b573703f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:30.525129 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce592f68-3d1c-4c7b-b5dd-b9d6b573703f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:30.565722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce592f68-3d1c-4c7b-b5dd-b9d6b573703f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:30.571595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-ce592f68-3d1c-4c7b-b5dd-b9d6b573703f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:30.571595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:30.571595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:30.571595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:30.571595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:30.571595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:30.571595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:30.571595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:30.571595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:30.571595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:30.571595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:30.571595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:30.571595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:30.571595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:30.571595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:30.571595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:30.572556 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:30.572556 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:30.572556 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:30.572556 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:30.572556 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:30.572556 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:30.572556 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:30.572556 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:30.572556 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:30.572556 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:30.572556 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:30.572556 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:30.572556 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:30.572556 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:30.572556 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:30.572556 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:30.573414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:30.573414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:30.573414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:30.573414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:30.573414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:30.573414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:30.573414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:30.573414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:30.573414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:30.573414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:30.573414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:30.573414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:30.579294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d0c6d-4b94-434b-8cdd-37ce1f4ec8f4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:30.583232 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 368/722] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Sep 1 16:18:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:30.707240 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a45be-b0db-40cf-a94b-a27ae84401f0 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:30.707567 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a45be-b0db-40cf-a94b-a27ae84401f0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:30.707801 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a45be-b0db-40cf-a94b-a27ae84401f0 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:30.709130 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 369/723] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:18:30.715694 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4534427-c9d8-4466-8afe-5216ddcae06a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:30.715982 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4534427-c9d8-4466-8afe-5216ddcae06a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:30.716261 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4534427-c9d8-4466-8afe-5216ddcae06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:30.749493 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b4534427-c9d8-4466-8afe-5216ddcae06a None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:30.750717 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b4534427-c9d8-4466-8afe-5216ddcae06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:30.784457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:30.787447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d0c6d-4b94-434b-8cdd-37ce1f4ec8f4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:30.858017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d83f5-ed56-44ec-956d-40f9980b28e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:30.858668 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d83f5-ed56-44ec-956d-40f9980b28e8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:30.862218 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d83f5-ed56-44ec-956d-40f9980b28e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:30.862218 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d83f5-ed56-44ec-956d-40f9980b28e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:30.862218 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d83f5-ed56-44ec-956d-40f9980b28e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:30.862218 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d83f5-ed56-44ec-956d-40f9980b28e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:30.862218 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d83f5-ed56-44ec-956d-40f9980b28e8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:30.883115 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 355/724] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Sep 1 16:18:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:30.884464 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d83f5-ed56-44ec-956d-40f9980b28e8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:30.900646 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 370/725] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:18:30.952522 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9952448-2402-4e03-97fa-29f0cdc3dffd None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:30.952863 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9952448-2402-4e03-97fa-29f0cdc3dffd None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:30.953086 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9952448-2402-4e03-97fa-29f0cdc3dffd None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:30.954603 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 371/726] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:18:30.960355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e269ed7-4712-4439-ad11-ea83e29a5cf3 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:30.960607 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e269ed7-4712-4439-ad11-ea83e29a5cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:30.960884 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e269ed7-4712-4439-ad11-ea83e29a5cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:30.994495 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7e269ed7-4712-4439-ad11-ea83e29a5cf3 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:30.995698 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7e269ed7-4712-4439-ad11-ea83e29a5cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:31.044908 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d83f5-ed56-44ec-956d-40f9980b28e8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:31.118684 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:31.145241 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecfcf32-b637-47ab-aec9-51c8b1532812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:31.146365 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfcf32-b637-47ab-aec9-51c8b1532812 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:31.146920 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfcf32-b637-47ab-aec9-51c8b1532812 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:31.147403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfcf32-b637-47ab-aec9-51c8b1532812 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:31.148919 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 372/727] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:18:31.187062 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecfcf32-b637-47ab-aec9-51c8b1532812 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:31.188837 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecfcf32-b637-47ab-aec9-51c8b1532812 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:31.212787 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-2ecfcf32-b637-47ab-aec9-51c8b1532812 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:31.212787 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:31.212787 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:31.212787 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:31.212787 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:31.212787 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:31.212787 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:31.212787 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:31.212787 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:31.212787 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:31.212787 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:31.212787 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:31.212787 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:31.212787 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:31.212787 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:31.212787 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:31.213850 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:31.213850 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:31.213850 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:31.213850 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:31.213850 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:31.213850 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:31.213850 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:31.213850 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:31.213850 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:31.213850 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:31.213850 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:31.213850 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:31.213850 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:31.213850 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:31.213850 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:31.213850 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:31.213850 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:31.219120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:31.219120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:31.219120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:31.220529 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 356/728] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:31.226613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e269ed7-4712-4439-ad11-ea83e29a5cf3 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:31.394947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e269ed7-4712-4439-ad11-ea83e29a5cf3 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:31.405267 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecfcf32-b637-47ab-aec9-51c8b1532812 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:31.435342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f06a8-c1ad-4c9a-ac64-ea8caaf67ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:31.435960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f06a8-c1ad-4c9a-ac64-ea8caaf67ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:31.439116 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f06a8-c1ad-4c9a-ac64-ea8caaf67ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:31.439404 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f06a8-c1ad-4c9a-ac64-ea8caaf67ed3 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:31.440070 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f06a8-c1ad-4c9a-ac64-ea8caaf67ed3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:31.440375 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f06a8-c1ad-4c9a-ac64-ea8caaf67ed3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:31.441867 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f06a8-c1ad-4c9a-ac64-ea8caaf67ed3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:31.446703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:31.474636 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 373/729] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:31.485441 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f06a8-c1ad-4c9a-ac64-ea8caaf67ed3 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:31.546579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:31.556394 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecfcf32-b637-47ab-aec9-51c8b1532812 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:31.592406 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170ba3b-1944-4e95-b5b2-37cc2f497698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:31.593146 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170ba3b-1944-4e95-b5b2-37cc2f497698 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:31.593435 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170ba3b-1944-4e95-b5b2-37cc2f497698 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:31.593711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170ba3b-1944-4e95-b5b2-37cc2f497698 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:31.596390 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f06a8-c1ad-4c9a-ac64-ea8caaf67ed3 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:31.603922 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3170ba3b-1944-4e95-b5b2-37cc2f497698 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:31.609114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-3170ba3b-1944-4e95-b5b2-37cc2f497698 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:18:31.609114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:31.609114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:31.609114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:31.609114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:31.609114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:31.609114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:31.609114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:31.609114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:31.609114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:31.609114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:31.609114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:31.609114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:31.609114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:31.609114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:31.609114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:31.610142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:31.610142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:31.610142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:31.610142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:31.610142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:31.610142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:31.610142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:31.610142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:31.610142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:31.610142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:31.610142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:31.610142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:31.610142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:31.610142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:31.610142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:31.610142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:31.611273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:31.611273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:31.611273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:31.611273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:31.611273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:31.611273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:31.611273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:31.611273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:31.611273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:31.611273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:31.611273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:18:31.611273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:31.611273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 357/730] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:18:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:18:31.611273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170ba3b-1944-4e95-b5b2-37cc2f497698 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:31.634388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551cd14-9edb-4009-a153-858db2f68c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:31.635013 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551cd14-9edb-4009-a153-858db2f68c5f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:31.635392 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551cd14-9edb-4009-a153-858db2f68c5f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:31.635560 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551cd14-9edb-4009-a153-858db2f68c5f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:31.642936 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551cd14-9edb-4009-a153-858db2f68c5f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:31.644366 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551cd14-9edb-4009-a153-858db2f68c5f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:31.657367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-9551cd14-9edb-4009-a153-858db2f68c5f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:18:31.657367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:31.657367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:31.657367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:31.657367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:31.657367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:31.657367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:31.657367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:31.657367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:31.657367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:31.657367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:31.657367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:31.657367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:31.657367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:31.657367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:31.657367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:31.658737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:31.658737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:31.658737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:31.658737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:31.658737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:31.658737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:31.658737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:31.658737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:31.658737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:31.658737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:31.658737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:31.658737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:31.658737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:31.658737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:31.658737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:31.658737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:31.658737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:31.658737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:31.659694 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:31.659694 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:31.659694 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:31.659694 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:31.659694 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:31.659694 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:31.659694 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:31.659694 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:31.659694 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:18:31.659694 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:31.662463 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 374/731] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Sep 1 16:18:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:31.663771 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91542442-1a11-4068-b8d7-5cee89cfbd44 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:31.664066 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91542442-1a11-4068-b8d7-5cee89cfbd44 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:31.664342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91542442-1a11-4068-b8d7-5cee89cfbd44 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:31.665586 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 375/732] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:31.666726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888a089-6345-48b7-a344-272e9eb88345 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:31.667365 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888a089-6345-48b7-a344-272e9eb88345 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:31.667646 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888a089-6345-48b7-a344-272e9eb88345 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:31.669001 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 376/733] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:31.670984 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888a089-6345-48b7-a344-272e9eb88345 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:31.686088 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:31.712007 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170ba3b-1944-4e95-b5b2-37cc2f497698 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:31.744205 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f95fd-54a2-4c22-9f3e-3a935a878189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:31.744756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f95fd-54a2-4c22-9f3e-3a935a878189 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:31.745036 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f95fd-54a2-4c22-9f3e-3a935a878189 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:31.745321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f95fd-54a2-4c22-9f3e-3a935a878189 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:31.755317 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f95fd-54a2-4c22-9f3e-3a935a878189 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:31.760990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-709f95fd-54a2-4c22-9f3e-3a935a878189 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:18:31.760990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:31.760990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:31.760990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:31.760990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:31.760990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:31.760990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:31.760990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:31.760990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:31.760990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:31.760990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:31.760990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:31.760990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:31.760990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:31.760990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:31.760990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:31.762123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:31.762123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:31.762123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:31.762123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:31.762123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:31.762123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:31.762123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:31.762123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:31.762123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:31.762123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:31.762123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:31.762123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:31.762123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:31.762123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:31.762123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:31.762123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:31.763451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:31.763451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:31.763451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:31.763451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:31.763451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:31.763451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:31.763451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:31.763451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:31.763451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:31.763451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:31.763451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:18:31.763451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:31.763451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 358/734] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:18:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:31.763451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f95fd-54a2-4c22-9f3e-3a935a878189 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:31.774963 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888a089-6345-48b7-a344-272e9eb88345 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:31.781912 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:31.810551 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d41f5a-51dc-47d3-a5f7-aee84a25e89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:31.811150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d41f5a-51dc-47d3-a5f7-aee84a25e89b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:31.811439 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d41f5a-51dc-47d3-a5f7-aee84a25e89b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:31.811710 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d41f5a-51dc-47d3-a5f7-aee84a25e89b None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:31.812376 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d41f5a-51dc-47d3-a5f7-aee84a25e89b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:31.812679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d41f5a-51dc-47d3-a5f7-aee84a25e89b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:31.814033 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d41f5a-51dc-47d3-a5f7-aee84a25e89b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:31.821914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 377/735] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:18:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:31.823146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f15bbf-e559-4ef5-87b8-0e9220ad6e47 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:31.823434 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f15bbf-e559-4ef5-87b8-0e9220ad6e47 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:31.823712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f15bbf-e559-4ef5-87b8-0e9220ad6e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:31.840685 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f8f15bbf-e559-4ef5-87b8-0e9220ad6e47 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:31.841859 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f8f15bbf-e559-4ef5-87b8-0e9220ad6e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:31.865013 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f95fd-54a2-4c22-9f3e-3a935a878189 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:31.898774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bc029-faba-4d40-9b46-d90973e965b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:31.899379 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bc029-faba-4d40-9b46-d90973e965b8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:31.899660 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bc029-faba-4d40-9b46-d90973e965b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:31.899941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bc029-faba-4d40-9b46-d90973e965b8 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:31.900669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bc029-faba-4d40-9b46-d90973e965b8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:31.900975 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bc029-faba-4d40-9b46-d90973e965b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:31.902449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bc029-faba-4d40-9b46-d90973e965b8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:31.904145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:31.907123 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 378/736] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:18:31.908328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4514f-52f4-4e90-8d91-b774198002e8 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:31.908608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4514f-52f4-4e90-8d91-b774198002e8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:31.908876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4514f-52f4-4e90-8d91-b774198002e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:31.920864 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 359/737] 10.222.0.58 () {56 vars in 1149 bytes} [Thu Sep 1 16:18:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:31.921668 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bc029-faba-4d40-9b46-d90973e965b8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:31.940896 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a5d4514f-52f4-4e90-8d91-b774198002e8 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:31.942328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a5d4514f-52f4-4e90-8d91-b774198002e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:32.018464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 379/738] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:18:32.018464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4514f-52f4-4e90-8d91-b774198002e8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:32.029701 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bc029-faba-4d40-9b46-d90973e965b8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:32.060077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:32.082385 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1c850-7653-4579-94ab-9493239c01b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:32.083038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1c850-7653-4579-94ab-9493239c01b3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:32.083322 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1c850-7653-4579-94ab-9493239c01b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:32.083595 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1c850-7653-4579-94ab-9493239c01b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:32.084251 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e1c850-7653-4579-94ab-9493239c01b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:32.084551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e1c850-7653-4579-94ab-9493239c01b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:32.086111 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e1c850-7653-4579-94ab-9493239c01b3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:32.099355 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 360/739] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:18:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:32.100216 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1c850-7653-4579-94ab-9493239c01b3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:32.156282 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4514f-52f4-4e90-8d91-b774198002e8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:32.168773 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:32.193098 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10df1c-5222-4dad-94cc-e43f31a260cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:32.194105 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10df1c-5222-4dad-94cc-e43f31a260cf None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:32.194427 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10df1c-5222-4dad-94cc-e43f31a260cf None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:32.195027 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10df1c-5222-4dad-94cc-e43f31a260cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:32.205639 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10df1c-5222-4dad-94cc-e43f31a260cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:32.207098 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10df1c-5222-4dad-94cc-e43f31a260cf None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:32.218655 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1c850-7653-4579-94ab-9493239c01b3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:32.221785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-ab10df1c-5222-4dad-94cc-e43f31a260cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:32.221785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:32.221785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:32.221785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:32.221785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:32.221785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:32.221785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:32.221785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:32.221785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:32.221785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:32.221785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:32.221785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:32.221785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:32.221785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:32.221785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:32.221785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:32.222775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:32.222775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:32.222775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:32.222775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:32.222775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:32.222775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:32.222775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:32.222775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:32.222775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:32.222775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:32.222775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:32.222775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:32.222775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:32.222775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:32.222775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:32.222775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:32.222775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:32.223720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:32.223720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:32.223720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:32.223720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 380/740] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:32.224413 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10df1c-5222-4dad-94cc-e43f31a260cf None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:32.260317 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb626cd-a016-4c72-8a02-48150af66578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:32.260923 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb626cd-a016-4c72-8a02-48150af66578 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:32.261207 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb626cd-a016-4c72-8a02-48150af66578 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:32.261646 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb626cd-a016-4c72-8a02-48150af66578 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:32.307925 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:32.328135 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10df1c-5222-4dad-94cc-e43f31a260cf None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:32.336162 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb626cd-a016-4c72-8a02-48150af66578 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:32.337630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb626cd-a016-4c72-8a02-48150af66578 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:32.362162 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d7439-4e3f-4d0e-839c-52eb053f21bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:32.362809 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d7439-4e3f-4d0e-839c-52eb053f21bc None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:32.363090 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d7439-4e3f-4d0e-839c-52eb053f21bc None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:32.363366 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d7439-4e3f-4d0e-839c-52eb053f21bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:32.373542 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732d7439-4e3f-4d0e-839c-52eb053f21bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:32.374995 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732d7439-4e3f-4d0e-839c-52eb053f21bc None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:32.388789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-732d7439-4e3f-4d0e-839c-52eb053f21bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:32.388789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:32.388789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:32.388789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:32.388789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:32.388789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:32.388789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:32.388789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:32.388789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:32.388789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:32.388789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:32.388789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:32.388789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:32.388789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:32.388789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:32.388789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:32.389763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:32.389763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:32.389763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:32.389763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:32.389763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:32.389763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:32.389763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:32.389763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:32.389763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:32.389763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:32.389763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:32.389763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:32.389763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:32.389763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:32.389763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:32.389763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:32.389763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:32.395580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:32.395580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:32.395580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:32.395792 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:32.395869 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 381/741] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:32.395869 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d7439-4e3f-4d0e-839c-52eb053f21bc None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:32.436520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 361/742] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:18:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:18:32.437345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb626cd-a016-4c72-8a02-48150af66578 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:32.494674 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d7439-4e3f-4d0e-839c-52eb053f21bc None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:32.523098 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:32.528440 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1f9bf-1caf-4f0b-b6e0-7750cad113c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:32.529027 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1f9bf-1caf-4f0b-b6e0-7750cad113c9 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:32.529307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1f9bf-1caf-4f0b-b6e0-7750cad113c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:32.529578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1f9bf-1caf-4f0b-b6e0-7750cad113c9 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:32.530247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca1f9bf-1caf-4f0b-b6e0-7750cad113c9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:32.531596 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca1f9bf-1caf-4f0b-b6e0-7750cad113c9 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:32.535913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb626cd-a016-4c72-8a02-48150af66578 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:32.547310 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 382/743] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Sep 1 16:18:32 2022] POST /identity/v3/services => generated 251 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:18:32.548089 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1f9bf-1caf-4f0b-b6e0-7750cad113c9 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:32.586897 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561fcb9-9b26-48ef-bb1b-7597d0efc742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:32.587733 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561fcb9-9b26-48ef-bb1b-7597d0efc742 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:32.588027 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561fcb9-9b26-48ef-bb1b-7597d0efc742 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:32.588370 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561fcb9-9b26-48ef-bb1b-7597d0efc742 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:32.589203 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561fcb9-9b26-48ef-bb1b-7597d0efc742 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:32.589907 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561fcb9-9b26-48ef-bb1b-7597d0efc742 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:32.591702 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561fcb9-9b26-48ef-bb1b-7597d0efc742 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:32.606045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 362/744] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:32.607287 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561fcb9-9b26-48ef-bb1b-7597d0efc742 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:32.621233 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:32.674310 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1f9bf-1caf-4f0b-b6e0-7750cad113c9 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:32.713357 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92d565-637d-4784-9efa-084b69fa4bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:32.714284 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92d565-637d-4784-9efa-084b69fa4bff None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:32.714284 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92d565-637d-4784-9efa-084b69fa4bff None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:32.714541 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92d565-637d-4784-9efa-084b69fa4bff None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:32.715195 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92d565-637d-4784-9efa-084b69fa4bff None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:32.716565 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92d565-637d-4784-9efa-084b69fa4bff None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:32.733005 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561fcb9-9b26-48ef-bb1b-7597d0efc742 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:32.733524 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 383/745] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Sep 1 16:18:32 2022] POST /identity/v3/services => generated 240 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:18:32.734404 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92d565-637d-4784-9efa-084b69fa4bff None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:32.788350 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d80c5-b7a6-420b-93b4-9089515102c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:32.789182 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d80c5-b7a6-420b-93b4-9089515102c9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:32.789623 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d80c5-b7a6-420b-93b4-9089515102c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:32.789996 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d80c5-b7a6-420b-93b4-9089515102c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:32.790806 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d80c5-b7a6-420b-93b4-9089515102c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:32.791250 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d80c5-b7a6-420b-93b4-9089515102c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:32.793939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d80c5-b7a6-420b-93b4-9089515102c9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:32.811496 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:32.814362 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 363/746] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:32.815491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d80c5-b7a6-420b-93b4-9089515102c9 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:32.884795 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92d565-637d-4784-9efa-084b69fa4bff None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:32.909837 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:32.934316 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a82e3-44a4-4204-a260-c6651c303acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:32.934986 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a82e3-44a4-4204-a260-c6651c303acc None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:32.935227 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a82e3-44a4-4204-a260-c6651c303acc None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:32.935499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a82e3-44a4-4204-a260-c6651c303acc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:32.947375 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a82e3-44a4-4204-a260-c6651c303acc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:32.948820 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a82e3-44a4-4204-a260-c6651c303acc None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:32.959414 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d80c5-b7a6-420b-93b4-9089515102c9 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:32.966800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-e51a82e3-44a4-4204-a260-c6651c303acc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:32.966800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:32.966800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:32.966800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:32.966800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:32.966800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:32.966800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:32.966800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:32.966800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:32.966800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:32.966800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:32.966800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:32.966800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:32.966800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:32.966800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:32.966800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:32.967889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:32.967889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:32.967889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:32.967889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:32.967889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:32.967889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:32.967889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:32.967889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:32.967889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:32.967889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:32.967889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:32.967889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:32.967889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:32.967889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:32.967889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:32.967889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:32.967889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:32.968808 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:32.968808 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:32.968808 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:32.968808 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 384/747] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:32.969957 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a82e3-44a4-4204-a260-c6651c303acc None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:33.000063 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441c5b1-c473-4698-8d0e-f72c6a369606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:33.000705 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441c5b1-c473-4698-8d0e-f72c6a369606 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:33.001155 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441c5b1-c473-4698-8d0e-f72c6a369606 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:33.001438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441c5b1-c473-4698-8d0e-f72c6a369606 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:33.011515 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6441c5b1-c473-4698-8d0e-f72c6a369606 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:33.013976 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6441c5b1-c473-4698-8d0e-f72c6a369606 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:33.028891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-6441c5b1-c473-4698-8d0e-f72c6a369606 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:18:33.028891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:33.028891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:33.028891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:33.028891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:33.028891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:33.028891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:33.028891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:33.028891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:33.028891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:33.028891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:33.028891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:33.028891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:33.028891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:33.028891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:33.028891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:33.030967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:33.030967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:33.030967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:33.030967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:33.030967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:33.030967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:33.030967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:33.030967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:33.030967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:33.030967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:33.030967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:33.030967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:33.030967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:33.030967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:33.030967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:33.030967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:33.030967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:33.030967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:33.033043 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:33.033043 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:33.033043 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:33.033043 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:33.033043 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:33.033043 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:33.033043 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:33.033043 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:33.033043 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:18:33.033043 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:33.033043 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 364/748] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Sep 1 16:18:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:33.033043 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441c5b1-c473-4698-8d0e-f72c6a369606 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:33.096611 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a82e3-44a4-4204-a260-c6651c303acc None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:33.108508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:33.141136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6bff3d-f9d9-4997-8b64-9d1b0aebf796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:33.141785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6bff3d-f9d9-4997-8b64-9d1b0aebf796 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:33.142460 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6bff3d-f9d9-4997-8b64-9d1b0aebf796 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:33.142815 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6bff3d-f9d9-4997-8b64-9d1b0aebf796 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:33.154086 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6bff3d-f9d9-4997-8b64-9d1b0aebf796 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:33.156713 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6bff3d-f9d9-4997-8b64-9d1b0aebf796 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:33.169019 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441c5b1-c473-4698-8d0e-f72c6a369606 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:33.172217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-ba6bff3d-f9d9-4997-8b64-9d1b0aebf796 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:18:33.172217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:33.172217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:33.172217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:33.172217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:33.172217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:33.172217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:33.172217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:33.172217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:33.172217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:33.172217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:33.172217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:33.172217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:33.172217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:33.172217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:33.172217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:33.174121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:33.174121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:33.174121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:33.174121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:33.174121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:33.174121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:33.174121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:33.174121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:33.174121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:33.174121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:33.174121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:33.174121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:33.174121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:33.174121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:33.174121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:33.174121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:33.174121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:33.174121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:33.175608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:33.175608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:33.175608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:33.175608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:33.175608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:33.175608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:33.175608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:33.175608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:33.175608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:18:33.175608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:33.175608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 385/749] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:18:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:33.175608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6bff3d-f9d9-4997-8b64-9d1b0aebf796 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:33.203746 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:33.218177 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8621b78-6d2a-41af-a975-f2ec45a14f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:33.219940 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8621b78-6d2a-41af-a975-f2ec45a14f45 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:33.220403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8621b78-6d2a-41af-a975-f2ec45a14f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:33.220830 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8621b78-6d2a-41af-a975-f2ec45a14f45 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:33.221639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8621b78-6d2a-41af-a975-f2ec45a14f45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:33.222095 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8621b78-6d2a-41af-a975-f2ec45a14f45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:33.224152 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8621b78-6d2a-41af-a975-f2ec45a14f45 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:33.239521 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 365/750] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:33.240841 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8621b78-6d2a-41af-a975-f2ec45a14f45 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:33.294078 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6bff3d-f9d9-4997-8b64-9d1b0aebf796 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:33.331261 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec6b20-517f-42df-9a77-4452bd6a42ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:33.332156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec6b20-517f-42df-9a77-4452bd6a42ec None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:33.332612 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec6b20-517f-42df-9a77-4452bd6a42ec None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:33.333032 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec6b20-517f-42df-9a77-4452bd6a42ec None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:33.333910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ec6b20-517f-42df-9a77-4452bd6a42ec None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:33.334470 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ec6b20-517f-42df-9a77-4452bd6a42ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:33.336834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ec6b20-517f-42df-9a77-4452bd6a42ec None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:33.344660 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8621b78-6d2a-41af-a975-f2ec45a14f45 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:33.360827 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 386/751] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Sep 1 16:18:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:33.362037 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec6b20-517f-42df-9a77-4452bd6a42ec None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:33.380031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6dec84-d3de-4146-bea0-3f02f8a8fdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:33.380576 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6dec84-d3de-4146-bea0-3f02f8a8fdba None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:33.380859 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6dec84-d3de-4146-bea0-3f02f8a8fdba None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:33.381134 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6dec84-d3de-4146-bea0-3f02f8a8fdba None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:33.381874 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6dec84-d3de-4146-bea0-3f02f8a8fdba None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:33.382163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6dec84-d3de-4146-bea0-3f02f8a8fdba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:33.384237 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6dec84-d3de-4146-bea0-3f02f8a8fdba None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:33.389728 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:33.402867 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 366/752] 10.222.0.58 () {56 vars in 1135 bytes} [Thu Sep 1 16:18:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:33.404217 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6dec84-d3de-4146-bea0-3f02f8a8fdba None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:33.452364 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:33.479952 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec6b20-517f-42df-9a77-4452bd6a42ec None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:33.510070 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6dec84-d3de-4146-bea0-3f02f8a8fdba None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:33.522111 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0941ba-29b2-49cf-8cdb-a5b6ccb13f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:33.523088 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0941ba-29b2-49cf-8cdb-a5b6ccb13f80 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:33.524037 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0941ba-29b2-49cf-8cdb-a5b6ccb13f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:33.524487 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0941ba-29b2-49cf-8cdb-a5b6ccb13f80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:33.532178 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0941ba-29b2-49cf-8cdb-a5b6ccb13f80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:33.533837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0941ba-29b2-49cf-8cdb-a5b6ccb13f80 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:33.547136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-fa0941ba-29b2-49cf-8cdb-a5b6ccb13f80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:33.547136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:33.547136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:33.547136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:33.547136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:33.547136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:33.547136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:33.547136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:33.547136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:33.547136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:33.547136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:33.547136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:33.547136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:33.547136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:33.547136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:33.548051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:33.548051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:33.548051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:33.548051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:33.548051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:33.548051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:33.548051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:33.548051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:33.548051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:33.548051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:33.548051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:33.548051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:33.548051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:33.548051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:33.548051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:33.548051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:33.548051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:33.548978 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:33.548978 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:33.548978 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:33.548978 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:33.548978 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:33.548978 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:33.548978 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:33.548978 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:33.550589 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e880a-202a-47db-9a2f-173329b13655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:33.551222 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 387/753] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:33.551553 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e880a-202a-47db-9a2f-173329b13655 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:33.552003 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e880a-202a-47db-9a2f-173329b13655 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:33.552375 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e880a-202a-47db-9a2f-173329b13655 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:33.552857 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0941ba-29b2-49cf-8cdb-a5b6ccb13f80 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:33.564131 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074e880a-202a-47db-9a2f-173329b13655 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:33.565704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074e880a-202a-47db-9a2f-173329b13655 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:33.581046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-074e880a-202a-47db-9a2f-173329b13655 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:33.581046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:33.581046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:33.581046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:33.581046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:33.581046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:33.581046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:33.581046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:33.581046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:33.581046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:33.581046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:33.581046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:33.581046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:33.581046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:33.581046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:33.581046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:33.582045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:33.582045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:33.582045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:33.582045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:33.582045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:33.582045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:33.582045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:33.582045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:33.582045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:33.582045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:33.582045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:33.582045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:33.582045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:33.582045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:33.582045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:33.582045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:33.582045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:33.583084 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:33.583084 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:33.583084 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:33.587146 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 367/754] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:33.588027 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e880a-202a-47db-9a2f-173329b13655 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:33.647616 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:33.671427 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0941ba-29b2-49cf-8cdb-a5b6ccb13f80 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:33.687098 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:33.693779 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e880a-202a-47db-9a2f-173329b13655 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:33.713916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce764f-0697-4e19-a516-0cca62bc5198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:33.714861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce764f-0697-4e19-a516-0cca62bc5198 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:33.715348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce764f-0697-4e19-a516-0cca62bc5198 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:33.715816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce764f-0697-4e19-a516-0cca62bc5198 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:33.731258 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f62f2d-e503-4940-bdcc-d426fb0f0d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:33.731963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f62f2d-e503-4940-bdcc-d426fb0f0d30 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:33.732316 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f62f2d-e503-4940-bdcc-d426fb0f0d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:33.732748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f62f2d-e503-4940-bdcc-d426fb0f0d30 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:33.733513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f62f2d-e503-4940-bdcc-d426fb0f0d30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:33.733934 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f62f2d-e503-4940-bdcc-d426fb0f0d30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:33.735492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f62f2d-e503-4940-bdcc-d426fb0f0d30 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:33.756806 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 368/755] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:33.758303 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f62f2d-e503-4940-bdcc-d426fb0f0d30 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:33.792886 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce764f-0697-4e19-a516-0cca62bc5198 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:33.794373 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce764f-0697-4e19-a516-0cca62bc5198 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:33.878397 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f62f2d-e503-4940-bdcc-d426fb0f0d30 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:33.878601 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:33.897444 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 388/756] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:18:33 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:18:33.898261 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce764f-0697-4e19-a516-0cca62bc5198 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:33.920998 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a88b2-99ae-4377-8977-6fa7b9c093da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:33.921974 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a88b2-99ae-4377-8977-6fa7b9c093da None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:33.923052 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a88b2-99ae-4377-8977-6fa7b9c093da None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:33.923517 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a88b2-99ae-4377-8977-6fa7b9c093da None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:33.924637 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a88b2-99ae-4377-8977-6fa7b9c093da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:33.925376 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a88b2-99ae-4377-8977-6fa7b9c093da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:33.927697 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a88b2-99ae-4377-8977-6fa7b9c093da None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:33.944457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 369/757] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:33.946789 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a88b2-99ae-4377-8977-6fa7b9c093da None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:33.971959 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:34.018812 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce764f-0697-4e19-a516-0cca62bc5198 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:34.058795 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9618a6-ec50-43d8-89c6-647902d762cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:34.059462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9618a6-ec50-43d8-89c6-647902d762cc None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:34.059748 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9618a6-ec50-43d8-89c6-647902d762cc None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:34.060025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9618a6-ec50-43d8-89c6-647902d762cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:34.063377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9618a6-ec50-43d8-89c6-647902d762cc None admin] RBAC: Adding query filter params (role.id=b0923c7a240149ae9ece5b3c1622e860, scope.project.id=479530ced580426b85d1fc07af43417f, user.id=dd7b98536fa445cebd552fcbf54b757d) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:34.063377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9618a6-ec50-43d8-89c6-647902d762cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:34.063377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9618a6-ec50-43d8-89c6-647902d762cc None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:34.078226 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 389/758] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:18:33 2022] GET /identity/v3/role_assignments?user.id=dd7b98536fa445cebd552fcbf54b757d&scope.project.id=479530ced580426b85d1fc07af43417f&role.id=b0923c7a240149ae9ece5b3c1622e860 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:34.078226 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9618a6-ec50-43d8-89c6-647902d762cc None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:34.131742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a88b2-99ae-4377-8977-6fa7b9c093da None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:34.151388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:34.176297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba533275-9475-456c-84eb-84c35445e23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:34.178036 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba533275-9475-456c-84eb-84c35445e23e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:34.178581 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba533275-9475-456c-84eb-84c35445e23e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:34.179063 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba533275-9475-456c-84eb-84c35445e23e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:34.193218 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba533275-9475-456c-84eb-84c35445e23e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:34.195863 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba533275-9475-456c-84eb-84c35445e23e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:34.214598 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-ba533275-9475-456c-84eb-84c35445e23e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:34.214598 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:34.214598 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:34.214598 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:34.214598 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:34.214598 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:34.214598 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:34.214598 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:34.214598 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:34.214598 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:34.214598 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:34.214598 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:34.214598 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:34.214598 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:34.214598 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:34.214598 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:34.215658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:34.215658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:34.215658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:34.215658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:34.215658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:34.215658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:34.215658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:34.215658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:34.215658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:34.215658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:34.215658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:34.215658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:34.215658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:34.215658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:34.215658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:34.215658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:34.215658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:34.216601 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:34.216601 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:34.216601 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:34.219450 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 370/759] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:34.225851 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba533275-9475-456c-84eb-84c35445e23e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:34.270910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9618a6-ec50-43d8-89c6-647902d762cc None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:34.317887 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b696470-86c2-4b4d-a9d5-dcbb32194abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:34.318918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b696470-86c2-4b4d-a9d5-dcbb32194abb None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:34.319428 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b696470-86c2-4b4d-a9d5-dcbb32194abb None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:34.320438 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b696470-86c2-4b4d-a9d5-dcbb32194abb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:34.332632 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b696470-86c2-4b4d-a9d5-dcbb32194abb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:34.334632 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b696470-86c2-4b4d-a9d5-dcbb32194abb None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:34.353832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-0b696470-86c2-4b4d-a9d5-dcbb32194abb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:34.353832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:34.353832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:34.353832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:34.353832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:34.353832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:34.353832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:34.353832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:34.353832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:34.353832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:34.353832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:34.353832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:34.353832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:34.353832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:34.353832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:34.354845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:34.354845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:34.354845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:34.354845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:34.354845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:34.354845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:34.354845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:34.354845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:34.354845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:34.354845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:34.354845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:34.354845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:34.354845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:34.354845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:34.354845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:34.354845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:34.354845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:34.355775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:34.355775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:34.355775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:34.355775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:34.355775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:34.355775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:34.355775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:34.355775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:34.363425 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 390/760] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:34.369695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b696470-86c2-4b4d-a9d5-dcbb32194abb None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:34.371765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:34.445393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba533275-9475-456c-84eb-84c35445e23e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:34.511894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f598db2-1fe3-45f8-9315-9b6e7583f164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:34.512893 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f598db2-1fe3-45f8-9315-9b6e7583f164 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:34.513394 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f598db2-1fe3-45f8-9315-9b6e7583f164 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:34.513890 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f598db2-1fe3-45f8-9315-9b6e7583f164 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:34.514797 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f598db2-1fe3-45f8-9315-9b6e7583f164 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:34.516557 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f598db2-1fe3-45f8-9315-9b6e7583f164 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:34.518139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f598db2-1fe3-45f8-9315-9b6e7583f164 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:34.543020 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 371/761] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:34.549368 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b696470-86c2-4b4d-a9d5-dcbb32194abb None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:34.550579 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f598db2-1fe3-45f8-9315-9b6e7583f164 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:34.561314 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:34.590811 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d36270e-fc46-4205-b433-c1a067f87ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:34.591885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36270e-fc46-4205-b433-c1a067f87ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:34.592589 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36270e-fc46-4205-b433-c1a067f87ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:34.593079 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36270e-fc46-4205-b433-c1a067f87ed3 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:34.593984 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d36270e-fc46-4205-b433-c1a067f87ed3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:34.594584 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d36270e-fc46-4205-b433-c1a067f87ed3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:34.596289 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d36270e-fc46-4205-b433-c1a067f87ed3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:34.610713 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 391/762] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:34.612336 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469650c2-d459-4cb0-a454-367d99a8fb3f None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:34.612829 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469650c2-d459-4cb0-a454-367d99a8fb3f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:34.613330 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469650c2-d459-4cb0-a454-367d99a8fb3f None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:34.615072 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 392/763] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:34.616145 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469650c2-d459-4cb0-a454-367d99a8fb3f None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:34.672815 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f598db2-1fe3-45f8-9315-9b6e7583f164 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:34.701378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:34.711538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915a3de-36ad-4f12-b74f-c03ff0227308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:34.712164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915a3de-36ad-4f12-b74f-c03ff0227308 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:34.712441 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915a3de-36ad-4f12-b74f-c03ff0227308 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:34.712712 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915a3de-36ad-4f12-b74f-c03ff0227308 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:34.720433 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f915a3de-36ad-4f12-b74f-c03ff0227308 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:34.721882 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f915a3de-36ad-4f12-b74f-c03ff0227308 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:34.724203 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469650c2-d459-4cb0-a454-367d99a8fb3f None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:34.734261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-f915a3de-36ad-4f12-b74f-c03ff0227308 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:34.734261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:34.734261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:34.734261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:34.734261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:34.734261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:34.734261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:34.734261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:34.734261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:34.734261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:34.734261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:34.734261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:34.734261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:34.734261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:34.734261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:34.735311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:34.735311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:34.735311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:34.735311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:34.735311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:34.735311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:34.735311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:34.735311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:34.735311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:34.735311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:34.735311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:34.735311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:34.735311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:34.735311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:34.735311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:34.735311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:34.735311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:34.736261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:34.736261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:34.736261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:34.736261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:34.736261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:34.736261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:34.736261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:34.736261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:34.740162 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 372/764] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:34.741483 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a5219-dafa-4943-ae71-0e759055cd15 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:34.741803 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a5219-dafa-4943-ae71-0e759055cd15 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:34.742080 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a5219-dafa-4943-ae71-0e759055cd15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:34.761387 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-455a5219-dafa-4943-ae71-0e759055cd15 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:34.762597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-455a5219-dafa-4943-ae71-0e759055cd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:34.765552 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d560a1-f4c8-4930-bbe8-822ae4bc8467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:34.766521 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d560a1-f4c8-4930-bbe8-822ae4bc8467 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:34.767055 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d560a1-f4c8-4930-bbe8-822ae4bc8467 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:34.767573 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d560a1-f4c8-4930-bbe8-822ae4bc8467 None admin] PATH_INFO: `/v3/projects/479530ced580426b85d1fc07af43417f/users/1f0c99fdf8bd4331ad7e641ee38554c1/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:34.769741 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d560a1-f4c8-4930-bbe8-822ae4bc8467 None admin] RBAC: Authorizing `identity:create_grant(project_id=479530ced580426b85d1fc07af43417f, user_id=1f0c99fdf8bd4331ad7e641ee38554c1, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:34.771479 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d560a1-f4c8-4930-bbe8-822ae4bc8467 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:34.788192 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 393/765] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Sep 1 16:18:34 2022] PUT /identity/v3/projects/479530ced580426b85d1fc07af43417f/users/1f0c99fdf8bd4331ad7e641ee38554c1/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:18:34.789523 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d560a1-f4c8-4930-bbe8-822ae4bc8467 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:34.821894 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:34.841649 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 373/766] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:18:34.865065 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab544b62-28cd-41fd-b12c-98bf2780334d None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:34.865506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab544b62-28cd-41fd-b12c-98bf2780334d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:34.865918 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab544b62-28cd-41fd-b12c-98bf2780334d None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:34.870318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 374/767] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:34.874170 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17e5fd-f530-432d-963b-b41fce514bd9 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:34.874473 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17e5fd-f530-432d-963b-b41fce514bd9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:34.874750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17e5fd-f530-432d-963b-b41fce514bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:34.903812 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0c17e5fd-f530-432d-963b-b41fce514bd9 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:34.906597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0c17e5fd-f530-432d-963b-b41fce514bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:34.921448 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d560a1-f4c8-4930-bbe8-822ae4bc8467 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:34.961401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed09b2-8f64-4c52-b9cf-f49c2a05471b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:34.962000 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed09b2-8f64-4c52-b9cf-f49c2a05471b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:34.962318 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed09b2-8f64-4c52-b9cf-f49c2a05471b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:34.962601 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed09b2-8f64-4c52-b9cf-f49c2a05471b None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:34.963364 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ed09b2-8f64-4c52-b9cf-f49c2a05471b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:34.963672 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ed09b2-8f64-4c52-b9cf-f49c2a05471b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:34.965111 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ed09b2-8f64-4c52-b9cf-f49c2a05471b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:34.982248 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 394/768] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:34.986007 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed09b2-8f64-4c52-b9cf-f49c2a05471b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:35.005366 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 375/769] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:18:35.081425 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:35.093501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed09b2-8f64-4c52-b9cf-f49c2a05471b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:35.128636 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726a67b-4095-45c7-be1f-dc1a3c77791d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:35.129237 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726a67b-4095-45c7-be1f-dc1a3c77791d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:35.129521 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726a67b-4095-45c7-be1f-dc1a3c77791d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:35.129791 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726a67b-4095-45c7-be1f-dc1a3c77791d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:35.130550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5726a67b-4095-45c7-be1f-dc1a3c77791d None admin] RBAC: Adding query filter params (role.id=ff293acb8551463aafe6970230042b89, scope.project.id=479530ced580426b85d1fc07af43417f, user.id=e789695517c1437fb224f5a6195655f9) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:35.130858 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5726a67b-4095-45c7-be1f-dc1a3c77791d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:35.132299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5726a67b-4095-45c7-be1f-dc1a3c77791d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:35.138065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 395/770] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:18:34 2022] GET /identity/v3/role_assignments?user.id=e789695517c1437fb224f5a6195655f9&scope.project.id=479530ced580426b85d1fc07af43417f&role.id=ff293acb8551463aafe6970230042b89 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:35.218696 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17e5fd-f530-432d-963b-b41fce514bd9 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:35.287611 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:35.359301 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17e5fd-f530-432d-963b-b41fce514bd9 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:35.393766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726a67b-4095-45c7-be1f-dc1a3c77791d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:35.412856 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95482e69-8440-4848-a6fb-a67e7e54cbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:35.414117 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95482e69-8440-4848-a6fb-a67e7e54cbeb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:35.414444 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95482e69-8440-4848-a6fb-a67e7e54cbeb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:35.414716 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95482e69-8440-4848-a6fb-a67e7e54cbeb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:35.415303 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95482e69-8440-4848-a6fb-a67e7e54cbeb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:35.416669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95482e69-8440-4848-a6fb-a67e7e54cbeb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:35.433580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-95482e69-8440-4848-a6fb-a67e7e54cbeb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:18:35.433580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:35.433580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:18:35.433580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:18:35.433580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:18:35.433580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:18:35.433580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:18:35.433580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:35.433580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:18:35.433580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:35.433580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:18:35.433580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:35.433580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:35.433580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:35.433580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:35.433580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:35.433580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:35.435952 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:35.435952 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:35.435952 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:35.435952 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:35.435952 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:35.435952 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:35.435952 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:18:35.435952 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:18:35.435952 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:18:35.435952 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:18:35.435952 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:35.435952 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:35.435952 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:35.435952 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:35.435952 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:35.435952 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:35.437950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:35.437950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:35.437950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:35.437950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:35.437950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:35.437950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:35.437950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:35.437950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:35.437950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:35.437950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:35.437950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:18:35.437950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:35.437950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:18:35.437950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:35.437950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 376/771] 10.222.0.58 () {56 vars in 1129 bytes} [Thu Sep 1 16:18:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:35.447191 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95482e69-8440-4848-a6fb-a67e7e54cbeb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:35.592441 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726a67b-4095-45c7-be1f-dc1a3c77791d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:35.609167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95482e69-8440-4848-a6fb-a67e7e54cbeb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:35.629895 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:35.664854 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bbd94-af61-4afd-b8a8-f1c74c1a96ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:35.665470 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bbd94-af61-4afd-b8a8-f1c74c1a96ca None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:35.665757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bbd94-af61-4afd-b8a8-f1c74c1a96ca None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:35.666041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bbd94-af61-4afd-b8a8-f1c74c1a96ca None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:35.666712 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006bbd94-af61-4afd-b8a8-f1c74c1a96ca None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:35.667656 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006bbd94-af61-4afd-b8a8-f1c74c1a96ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:35.669506 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea749df6-924e-4c84-8c8f-612e01c5aae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:35.670137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea749df6-924e-4c84-8c8f-612e01c5aae7 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:35.670449 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea749df6-924e-4c84-8c8f-612e01c5aae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:35.670721 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea749df6-924e-4c84-8c8f-612e01c5aae7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:35.671307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea749df6-924e-4c84-8c8f-612e01c5aae7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:35.672658 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea749df6-924e-4c84-8c8f-612e01c5aae7 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:35.673710 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006bbd94-af61-4afd-b8a8-f1c74c1a96ca None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:35.684990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 377/772] 10.222.0.58 () {56 vars in 1124 bytes} [Thu Sep 1 16:18:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:35.692731 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-ea749df6-924e-4c84-8c8f-612e01c5aae7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:18:35.692731 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:35.692731 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:18:35.692731 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:18:35.692731 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:18:35.692731 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:18:35.692731 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:18:35.692731 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:35.692731 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:18:35.692731 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:35.692731 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:18:35.692731 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:35.692731 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:35.692731 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:35.692731 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:35.692731 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:35.692731 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:35.695928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:35.695928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:35.695928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:35.695928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:35.695928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:35.695928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:35.695928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:18:35.695928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:18:35.695928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:18:35.695928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:18:35.695928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:35.695928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:35.695928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:35.695928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:35.695928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:35.695928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:35.698067 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:35.698067 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:35.698067 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:35.698067 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:35.698067 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:35.698067 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:35.698067 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:35.698067 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:35.698067 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:35.698067 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:35.698067 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:18:35.698067 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:35.698067 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:18:35.698067 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:35.698067 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 396/773] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:35 2022] GET /identity/v3/services/network => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:18:35.700207 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bbd94-af61-4afd-b8a8-f1c74c1a96ca None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:35.705238 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea749df6-924e-4c84-8c8f-612e01c5aae7 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:35.729929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:35.814518 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bbd94-af61-4afd-b8a8-f1c74c1a96ca None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:35.851888 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3258d-3764-452d-8b1b-dc5db7b4d02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:35.853153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3258d-3764-452d-8b1b-dc5db7b4d02c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:35.853936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3258d-3764-452d-8b1b-dc5db7b4d02c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:35.854515 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3258d-3764-452d-8b1b-dc5db7b4d02c None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:35.855364 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3258d-3764-452d-8b1b-dc5db7b4d02c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:35.855932 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3258d-3764-452d-8b1b-dc5db7b4d02c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:35.857866 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3258d-3764-452d-8b1b-dc5db7b4d02c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:35.865505 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 378/774] 10.222.0.58 () {56 vars in 1124 bytes} [Thu Sep 1 16:18:35 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:18:35.874010 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3258d-3764-452d-8b1b-dc5db7b4d02c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:35.923656 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:35.960582 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea749df6-924e-4c84-8c8f-612e01c5aae7 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:35.988837 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3258d-3764-452d-8b1b-dc5db7b4d02c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:36.027537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e3c59-9514-452e-8af2-7dee1deb24da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:36.028736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e3c59-9514-452e-8af2-7dee1deb24da None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:36.029207 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e3c59-9514-452e-8af2-7dee1deb24da None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:36.029666 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e3c59-9514-452e-8af2-7dee1deb24da None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:36.030602 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8e3c59-9514-452e-8af2-7dee1deb24da None admin] RBAC: Adding query filter params (interface=public, service_id=dc03e2c042ad48b78dcca18cf504bad8, region_id=RegionOne) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:36.031117 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8e3c59-9514-452e-8af2-7dee1deb24da None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:36.032786 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8e3c59-9514-452e-8af2-7dee1deb24da None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:36.046322 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 379/775] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Sep 1 16:18:35 2022] GET /identity/v3/endpoints?service_id=dc03e2c042ad48b78dcca18cf504bad8&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:36.048051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e3c59-9514-452e-8af2-7dee1deb24da None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:36.082944 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e10af1-a606-4de3-a147-07277fc90a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:36.083519 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e10af1-a606-4de3-a147-07277fc90a18 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:36.083800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e10af1-a606-4de3-a147-07277fc90a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:36.084132 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e10af1-a606-4de3-a147-07277fc90a18 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:36.084718 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e10af1-a606-4de3-a147-07277fc90a18 None admin] RBAC: Adding query filter params (name=network) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:36.085011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e10af1-a606-4de3-a147-07277fc90a18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:36.086485 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e10af1-a606-4de3-a147-07277fc90a18 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:36.102481 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 397/776] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Sep 1 16:18:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:36.111232 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e10af1-a606-4de3-a147-07277fc90a18 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:36.168405 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e3c59-9514-452e-8af2-7dee1deb24da None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:36.177217 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:36.244920 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27a07f-e554-410f-82d3-677a07273b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:36.246042 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27a07f-e554-410f-82d3-677a07273b90 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:36.246042 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27a07f-e554-410f-82d3-677a07273b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:36.246042 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27a07f-e554-410f-82d3-677a07273b90 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:36.246707 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e27a07f-e554-410f-82d3-677a07273b90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:36.249236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e27a07f-e554-410f-82d3-677a07273b90 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:36.263679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 380/777] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Sep 1 16:18:36 2022] GET /identity/v3/services => generated 772 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:36.303122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:36.340161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e10af1-a606-4de3-a147-07277fc90a18 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:36.391909 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e8146-5aec-43cb-90b7-b1b3fcef6bdb None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:36.392098 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e8146-5aec-43cb-90b7-b1b3fcef6bdb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:36.392379 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e8146-5aec-43cb-90b7-b1b3fcef6bdb None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:36.393904 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 381/778] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:36.395449 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a30fea-d71a-43d3-bf5a-e9fccf53af5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:36.396074 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a30fea-d71a-43d3-bf5a-e9fccf53af5e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:36.396356 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a30fea-d71a-43d3-bf5a-e9fccf53af5e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:36.396627 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a30fea-d71a-43d3-bf5a-e9fccf53af5e None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:36.397276 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a30fea-d71a-43d3-bf5a-e9fccf53af5e None admin] RBAC: Adding query filter params (type=network) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:36.397579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a30fea-d71a-43d3-bf5a-e9fccf53af5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:36.403666 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a30fea-d71a-43d3-bf5a-e9fccf53af5e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:36.407884 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8614d0-694f-44b6-b900-0a672a6491ad None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:36.408036 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8614d0-694f-44b6-b900-0a672a6491ad None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:36.408365 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8614d0-694f-44b6-b900-0a672a6491ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:36.413560 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 398/779] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Sep 1 16:18:36 2022] GET /identity/v3/services?type=network => generated 350 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:36.419287 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a30fea-d71a-43d3-bf5a-e9fccf53af5e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:36.428332 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6c8614d0-694f-44b6-b900-0a672a6491ad None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:36.429579 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6c8614d0-694f-44b6-b900-0a672a6491ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:36.504103 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 382/780] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:18:36.533389 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a30fea-d71a-43d3-bf5a-e9fccf53af5e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:36.573279 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6730135-4fb6-4505-b687-b4a131ea13cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:36.574295 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6730135-4fb6-4505-b687-b4a131ea13cf None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:36.574812 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6730135-4fb6-4505-b687-b4a131ea13cf None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:36.575291 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6730135-4fb6-4505-b687-b4a131ea13cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:36.576208 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6730135-4fb6-4505-b687-b4a131ea13cf None admin] RBAC: Adding query filter params (interface=public, service_id=3ac85e8bf4f34099a073f1f406442069, region_id=RegionOne) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:36.576723 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6730135-4fb6-4505-b687-b4a131ea13cf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:36.578318 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6730135-4fb6-4505-b687-b4a131ea13cf None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:36.588845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 399/781] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Sep 1 16:18:36 2022] GET /identity/v3/endpoints?service_id=3ac85e8bf4f34099a073f1f406442069&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:18:36.593955 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8614d0-694f-44b6-b900-0a672a6491ad None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:36.725060 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:36.728160 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8614d0-694f-44b6-b900-0a672a6491ad None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:36.764446 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7cc8a0-4a1d-4830-93f6-9165273d06a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:36.765060 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cc8a0-4a1d-4830-93f6-9165273d06a9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:36.765343 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cc8a0-4a1d-4830-93f6-9165273d06a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:36.765623 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cc8a0-4a1d-4830-93f6-9165273d06a9 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:36.766273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7cc8a0-4a1d-4830-93f6-9165273d06a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:36.767763 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7cc8a0-4a1d-4830-93f6-9165273d06a9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:36.773616 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 383/782] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Sep 1 16:18:36 2022] GET /identity/v3/services => generated 772 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:36.977819 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6730135-4fb6-4505-b687-b4a131ea13cf None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:36.987003 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:37.096477 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6730135-4fb6-4505-b687-b4a131ea13cf None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:37.108208 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d8302-8388-4348-af75-4f6bd29e2f13 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:37.108800 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d8302-8388-4348-af75-4f6bd29e2f13 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:37.109296 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d8302-8388-4348-af75-4f6bd29e2f13 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:37.111090 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 384/783] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:37.116095 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8d1b0-ae8a-4881-a8f5-14437dc3cc87 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:37.116524 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8d1b0-ae8a-4881-a8f5-14437dc3cc87 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:37.117462 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8d1b0-ae8a-4881-a8f5-14437dc3cc87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:37.134400 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96659a-36e3-4712-9194-131aa6f36460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:37.135368 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96659a-36e3-4712-9194-131aa6f36460 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:37.135784 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96659a-36e3-4712-9194-131aa6f36460 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:37.136199 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96659a-36e3-4712-9194-131aa6f36460 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:37.146861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96659a-36e3-4712-9194-131aa6f36460 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:37.152452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-9b96659a-36e3-4712-9194-131aa6f36460 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:37.152452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:37.152452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:37.152452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:37.152452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:37.152452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:37.152452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:37.152452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:37.152452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:37.152452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:37.152452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:37.152452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:37.152452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:37.152452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:37.152452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:37.152452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:37.153578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:37.153578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:37.153578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:37.153578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:37.153578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:37.153578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:37.153578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:37.153578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:37.153578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:37.153578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:37.153578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:37.153578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:37.153578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:37.153578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:37.153578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:37.153578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:37.154679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:37.154679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:37.154679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:37.154679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:37.154679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:37.154679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:37.154679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:37.154679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:37.154679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:37.154679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:37.154679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:37.154679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:37.157031 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 400/784] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Sep 1 16:18:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:37.158237 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-00b8d1b0-ae8a-4881-a8f5-14437dc3cc87 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:37.159732 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-00b8d1b0-ae8a-4881-a8f5-14437dc3cc87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:37.165477 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96659a-36e3-4712-9194-131aa6f36460 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:37.276529 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 385/785] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:18:37.286989 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:37.322266 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96659a-36e3-4712-9194-131aa6f36460 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:37.371828 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf7a94-9e2e-4ff1-a7ee-b85f3941420f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:37.372809 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf7a94-9e2e-4ff1-a7ee-b85f3941420f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:37.373294 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf7a94-9e2e-4ff1-a7ee-b85f3941420f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:37.373767 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf7a94-9e2e-4ff1-a7ee-b85f3941420f None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:37.374674 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf7a94-9e2e-4ff1-a7ee-b85f3941420f None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:37.375212 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf7a94-9e2e-4ff1-a7ee-b85f3941420f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:37.376852 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf7a94-9e2e-4ff1-a7ee-b85f3941420f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:37.390528 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 401/786] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Sep 1 16:18:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:18:37.395729 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8d1b0-ae8a-4881-a8f5-14437dc3cc87 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:37.498791 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:37.507812 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8d1b0-ae8a-4881-a8f5-14437dc3cc87 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:37.543937 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d7413-c96f-465c-b088-663f6b9da9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:37.544491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d7413-c96f-465c-b088-663f6b9da9a7 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:37.544775 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d7413-c96f-465c-b088-663f6b9da9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:37.545054 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d7413-c96f-465c-b088-663f6b9da9a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:37.555578 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d7413-c96f-465c-b088-663f6b9da9a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:37.557048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d7413-c96f-465c-b088-663f6b9da9a7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:37.572299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-985d7413-c96f-465c-b088-663f6b9da9a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:37.572299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:37.572299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:37.572299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:37.572299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:37.572299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:37.572299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:37.572299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:37.572299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:37.572299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:37.572299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:37.572299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:37.572299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:37.572299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:37.572299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:37.572299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:37.573333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:37.573333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:37.573333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:37.573333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:37.573333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:37.573333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:37.573333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:37.573333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:37.573333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:37.573333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:37.573333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:37.573333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:37.573333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:37.573333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:37.573333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:37.573333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:37.573333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:37.574299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:37.574299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:37.574299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:37.574299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 386/787] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:18:37.579404 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf7a94-9e2e-4ff1-a7ee-b85f3941420f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:37.589535 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d7413-c96f-465c-b088-663f6b9da9a7 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:37.748653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d7413-c96f-465c-b088-663f6b9da9a7 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:37.764898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf7a94-9e2e-4ff1-a7ee-b85f3941420f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:37.794812 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e809997-cda2-439b-92de-cc603a2e0526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:37.795834 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e809997-cda2-439b-92de-cc603a2e0526 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:37.796122 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e809997-cda2-439b-92de-cc603a2e0526 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:37.796659 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e809997-cda2-439b-92de-cc603a2e0526 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:37.798847 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:37.809309 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613644a3-53e3-4e7a-acd8-c6097e9adb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:37.809911 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613644a3-53e3-4e7a-acd8-c6097e9adb01 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:37.810208 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613644a3-53e3-4e7a-acd8-c6097e9adb01 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:37.810496 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613644a3-53e3-4e7a-acd8-c6097e9adb01 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:37.811145 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613644a3-53e3-4e7a-acd8-c6097e9adb01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:37.811590 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613644a3-53e3-4e7a-acd8-c6097e9adb01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:37.812386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e809997-cda2-439b-92de-cc603a2e0526 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:37.813116 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613644a3-53e3-4e7a-acd8-c6097e9adb01 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:37.820167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-8e809997-cda2-439b-92de-cc603a2e0526 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:18:37.820167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:37.820167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:37.820167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:37.820167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:37.820167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:37.820167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:37.820167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:37.820167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:37.820167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:37.820167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:37.820167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:37.820167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:37.820167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:37.820167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:37.820167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:37.821333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:37.821333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:37.821333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:37.821333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:37.821333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:37.821333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:37.821333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:37.821333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:37.821333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:37.821333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:37.821333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:37.821333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:37.821333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:37.821333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:37.821333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:37.821333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:37.822426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:37.822426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:37.822426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:37.822426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:37.822426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:37.822426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:37.822426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:37.822426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:37.822426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:37.822426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:37.822426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:18:37.822426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:37.825797 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 387/788] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:18:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:37.827259 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38134e-6be1-49fe-af64-cd5a6ed50840 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:37.827698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38134e-6be1-49fe-af64-cd5a6ed50840 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:37.828106 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38134e-6be1-49fe-af64-cd5a6ed50840 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:37.829736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 388/789] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:37.832356 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38134e-6be1-49fe-af64-cd5a6ed50840 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:37.839868 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 402/790] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:37.840023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e1592-3c1a-4b78-8f02-4abc0d931900 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:37.840341 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e1592-3c1a-4b78-8f02-4abc0d931900 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:37.840688 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e1592-3c1a-4b78-8f02-4abc0d931900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:37.842619 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:37.869134 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-291e1592-3c1a-4b78-8f02-4abc0d931900 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:37.871225 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-291e1592-3c1a-4b78-8f02-4abc0d931900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:37.958378 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 403/791] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:18:37.959635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e1592-3c1a-4b78-8f02-4abc0d931900 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:37.981538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38134e-6be1-49fe-af64-cd5a6ed50840 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:38.020021 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39609b-0dc6-4d62-aae5-0666b293cf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:38.020866 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39609b-0dc6-4d62-aae5-0666b293cf2b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:38.021225 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39609b-0dc6-4d62-aae5-0666b293cf2b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:38.021593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39609b-0dc6-4d62-aae5-0666b293cf2b None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:38.022371 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e39609b-0dc6-4d62-aae5-0666b293cf2b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:38.023295 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e39609b-0dc6-4d62-aae5-0666b293cf2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:38.026380 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e39609b-0dc6-4d62-aae5-0666b293cf2b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:38.035582 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 389/792] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:18:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:18:38.045383 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39609b-0dc6-4d62-aae5-0666b293cf2b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:38.159326 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e1592-3c1a-4b78-8f02-4abc0d931900 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:38.169481 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:38.184216 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:38.224987 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26a251-dcc2-43ed-ae5d-1c3d270ddfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:38.225625 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26a251-dcc2-43ed-ae5d-1c3d270ddfaa None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:38.225935 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26a251-dcc2-43ed-ae5d-1c3d270ddfaa None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:38.226222 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26a251-dcc2-43ed-ae5d-1c3d270ddfaa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:38.238260 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a26a251-dcc2-43ed-ae5d-1c3d270ddfaa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:38.239703 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a26a251-dcc2-43ed-ae5d-1c3d270ddfaa None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:38.246550 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39609b-0dc6-4d62-aae5-0666b293cf2b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:38.266367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-4a26a251-dcc2-43ed-ae5d-1c3d270ddfaa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:18:38.266367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:38.266367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:38.266367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:38.266367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:38.266367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:38.266367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:38.266367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:38.266367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:38.266367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:38.266367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:38.266367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:38.266367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:38.266367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:38.266367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:38.266367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:38.267377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:38.267377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:38.267377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:38.267377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:38.267377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:38.267377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:38.267377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:38.267377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:38.267377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:38.267377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:38.267377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:38.267377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:38.267377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:38.267377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:38.267377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:38.267377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:38.267377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:38.267377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:38.268364 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:38.268364 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:38.268364 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:38.268364 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:38.268364 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:38.268364 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:38.268364 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:38.268364 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:38.268364 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:18:38.268364 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:38.268364 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 404/793] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Sep 1 16:18:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:38.275704 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26a251-dcc2-43ed-ae5d-1c3d270ddfaa None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:38.333743 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97cee6e-6bd0-4360-8667-280fa5b98d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:38.334429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cee6e-6bd0-4360-8667-280fa5b98d51 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:38.334592 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cee6e-6bd0-4360-8667-280fa5b98d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:38.334857 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cee6e-6bd0-4360-8667-280fa5b98d51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:38.354482 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97cee6e-6bd0-4360-8667-280fa5b98d51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:38.355948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97cee6e-6bd0-4360-8667-280fa5b98d51 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:38.373016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-a97cee6e-6bd0-4360-8667-280fa5b98d51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:38.373016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:38.373016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:38.373016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:38.373016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:38.373016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:38.373016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:38.373016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:38.373016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:38.373016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:38.373016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:38.373016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:38.373016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:38.373016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:38.373016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:38.373016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:38.375275 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:38.375275 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:38.375275 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:38.375275 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:38.375275 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:38.375275 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:38.375275 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:38.375275 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:38.375275 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:38.375275 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:38.375275 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:38.375275 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:38.375275 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:38.375275 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:38.375275 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:38.375275 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:38.375275 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:38.377412 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:38.377412 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:38.377412 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:38.377412 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 390/794] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:38.377412 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97cee6e-6bd0-4360-8667-280fa5b98d51 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:38.405503 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26a251-dcc2-43ed-ae5d-1c3d270ddfaa None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:38.447229 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e17de-6d58-473c-a06b-dcf297affb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:38.447895 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e17de-6d58-473c-a06b-dcf297affb36 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:38.448175 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e17de-6d58-473c-a06b-dcf297affb36 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:38.448448 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e17de-6d58-473c-a06b-dcf297affb36 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:38.449258 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e17de-6d58-473c-a06b-dcf297affb36 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:38.449823 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e17de-6d58-473c-a06b-dcf297affb36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:38.451068 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e17de-6d58-473c-a06b-dcf297affb36 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:38.469749 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 405/795] 10.222.0.58 () {56 vars in 1149 bytes} [Thu Sep 1 16:18:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:38.470768 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e17de-6d58-473c-a06b-dcf297affb36 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:38.501459 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97cee6e-6bd0-4360-8667-280fa5b98d51 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:38.511304 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:38.522287 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:38.542476 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b47a8-4d54-412f-a55b-277425bea72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:38.543539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b47a8-4d54-412f-a55b-277425bea72e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:38.544026 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b47a8-4d54-412f-a55b-277425bea72e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:38.544502 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b47a8-4d54-412f-a55b-277425bea72e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:38.556156 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2b47a8-4d54-412f-a55b-277425bea72e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:38.561760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-3b2b47a8-4d54-412f-a55b-277425bea72e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:18:38.561760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:38.561760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:38.561760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:38.561760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:38.561760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:38.561760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:38.561760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:38.561760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:38.561760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:38.561760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:38.561760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:38.561760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:38.561760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:38.561760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:38.561760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:38.562730 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:38.562730 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:38.562730 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:38.562730 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:38.562730 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:38.562730 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:38.562730 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:38.562730 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:38.562730 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:38.562730 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:38.562730 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:38.562730 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:38.562730 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:38.562730 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:38.562730 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:38.562730 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:38.563599 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:38.563599 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:38.563599 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:38.563599 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:38.563599 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:38.563599 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:38.563599 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:38.563599 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:38.563599 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:38.563599 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:38.563599 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:18:38.563599 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:38.566154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 391/796] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:18:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:38.567157 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b47a8-4d54-412f-a55b-277425bea72e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:38.589443 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e17de-6d58-473c-a06b-dcf297affb36 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:38.625702 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0d78d-dffd-4980-b204-efb45e6bdc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:38.626400 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0d78d-dffd-4980-b204-efb45e6bdc17 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:38.626684 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0d78d-dffd-4980-b204-efb45e6bdc17 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:38.627145 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0d78d-dffd-4980-b204-efb45e6bdc17 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:38.627605 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0d78d-dffd-4980-b204-efb45e6bdc17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:38.627905 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0d78d-dffd-4980-b204-efb45e6bdc17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:38.629288 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0d78d-dffd-4980-b204-efb45e6bdc17 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:38.641958 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 406/797] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:38.643095 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0d78d-dffd-4980-b204-efb45e6bdc17 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:38.677898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b47a8-4d54-412f-a55b-277425bea72e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:38.714874 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86486a3-09c6-4028-9ed9-03df0ed76849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:38.715535 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86486a3-09c6-4028-9ed9-03df0ed76849 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:38.715825 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86486a3-09c6-4028-9ed9-03df0ed76849 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:38.716102 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86486a3-09c6-4028-9ed9-03df0ed76849 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:38.729051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86486a3-09c6-4028-9ed9-03df0ed76849 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:38.730604 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86486a3-09c6-4028-9ed9-03df0ed76849 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:38.746378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-d86486a3-09c6-4028-9ed9-03df0ed76849 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:38.746378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:38.746378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:38.746378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:38.746378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:38.746378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:38.746378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:38.746378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:38.746378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:38.746378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:38.746378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:38.746378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:38.746378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:38.746378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:38.746378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:38.746378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:38.747393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:38.747393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:38.747393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:38.747393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:38.747393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:38.747393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:38.747393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:38.747393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:38.747393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:38.747393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:38.747393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:38.747393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:38.747393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:38.747393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:38.747393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:38.747393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:38.747393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:38.748394 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:38.748394 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:38.748394 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:38.748394 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 392/798] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:38.749132 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584540a4-0847-46ef-8beb-1dd6cad786fa None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:38.749132 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584540a4-0847-46ef-8beb-1dd6cad786fa None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:38.749259 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584540a4-0847-46ef-8beb-1dd6cad786fa None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:38.752423 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 393/799] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:38.752423 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584540a4-0847-46ef-8beb-1dd6cad786fa None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:38.758698 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:38.764668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0d78d-dffd-4980-b204-efb45e6bdc17 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:38.787051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:38.801473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc503db-4ed6-428a-b2c9-e8c2e1c25978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:38.802400 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc503db-4ed6-428a-b2c9-e8c2e1c25978 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:38.802905 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc503db-4ed6-428a-b2c9-e8c2e1c25978 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:38.803378 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc503db-4ed6-428a-b2c9-e8c2e1c25978 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:38.804772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc503db-4ed6-428a-b2c9-e8c2e1c25978 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:38.805256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc503db-4ed6-428a-b2c9-e8c2e1c25978 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:38.806867 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc503db-4ed6-428a-b2c9-e8c2e1c25978 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:38.817042 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 407/800] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:18:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:38.818137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc503db-4ed6-428a-b2c9-e8c2e1c25978 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:38.869380 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584540a4-0847-46ef-8beb-1dd6cad786fa None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:38.907635 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38efec43-8575-4313-af6a-8cfa2866273a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:38.908291 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efec43-8575-4313-af6a-8cfa2866273a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:38.908641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efec43-8575-4313-af6a-8cfa2866273a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:38.908922 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efec43-8575-4313-af6a-8cfa2866273a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:38.918745 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38efec43-8575-4313-af6a-8cfa2866273a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:38.920133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38efec43-8575-4313-af6a-8cfa2866273a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:38.932775 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-38efec43-8575-4313-af6a-8cfa2866273a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:18:38.932775 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:38.932775 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:38.932775 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:38.932775 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:38.932775 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:38.932775 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:38.932775 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:38.932775 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:38.932775 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:38.932775 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:38.932775 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:38.932775 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:38.932775 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:38.932775 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:38.932775 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:38.933842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:38.933842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:38.933842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:38.933842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:38.933842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:38.933842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:38.933842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:38.933842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:38.933842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:38.933842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:38.933842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:38.933842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:38.933842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:38.933842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:38.933842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:38.933842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:38.933842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:38.933842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:38.935252 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:38.935252 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:38.935252 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:38.935252 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:38.935252 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:38.935252 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:38.935252 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:38.935252 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:38.935252 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:18:38.935252 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:38.938634 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 394/801] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Sep 1 16:18:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:38.940335 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b0867-ccc6-46a1-ac76-af13e20a486d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:38.940623 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b0867-ccc6-46a1-ac76-af13e20a486d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:38.941019 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b0867-ccc6-46a1-ac76-af13e20a486d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:38.961003 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-672b0867-ccc6-46a1-ac76-af13e20a486d None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:38.962256 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-672b0867-ccc6-46a1-ac76-af13e20a486d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:39.062214 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 395/802] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:18:39.062214 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b0867-ccc6-46a1-ac76-af13e20a486d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:39.069117 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc503db-4ed6-428a-b2c9-e8c2e1c25978 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:39.082990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:39.105455 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194584f1-23ad-43be-ab20-ad56b2700d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:39.106046 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194584f1-23ad-43be-ab20-ad56b2700d5f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:39.106351 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194584f1-23ad-43be-ab20-ad56b2700d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:39.106615 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194584f1-23ad-43be-ab20-ad56b2700d5f None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:39.107258 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194584f1-23ad-43be-ab20-ad56b2700d5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:39.107555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194584f1-23ad-43be-ab20-ad56b2700d5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:39.108898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194584f1-23ad-43be-ab20-ad56b2700d5f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:39.125139 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 408/803] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:39.125139 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194584f1-23ad-43be-ab20-ad56b2700d5f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:39.139481 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:39.176222 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b0867-ccc6-46a1-ac76-af13e20a486d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:39.212233 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a633b08-e98b-494a-a6c1-de9e10cc7f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:39.213023 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a633b08-e98b-494a-a6c1-de9e10cc7f09 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:39.213308 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a633b08-e98b-494a-a6c1-de9e10cc7f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:39.213679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a633b08-e98b-494a-a6c1-de9e10cc7f09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:39.222989 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194584f1-23ad-43be-ab20-ad56b2700d5f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:39.224017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a633b08-e98b-494a-a6c1-de9e10cc7f09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:39.225667 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a633b08-e98b-494a-a6c1-de9e10cc7f09 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:39.245686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-9a633b08-e98b-494a-a6c1-de9e10cc7f09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:39.245686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:39.245686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:39.245686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:39.245686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:39.245686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:39.245686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:39.245686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:39.245686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:39.245686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:39.245686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:39.245686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:39.245686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:39.245686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:39.245686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:39.245686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:39.246750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:39.246750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:39.246750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:39.246750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:39.246750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:39.246750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:39.246750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:39.246750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:39.246750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:39.246750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:39.246750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:39.246750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:39.246750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:39.246750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:39.246750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:39.246750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:39.246750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:39.247666 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:39.247666 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:39.247666 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:39.250327 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 396/804] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:39.252591 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95623837-5056-4c80-981e-da5756760ad0 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:39.253147 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95623837-5056-4c80-981e-da5756760ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:39.253429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95623837-5056-4c80-981e-da5756760ad0 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:39.255318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 397/805] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:39.258931 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95623837-5056-4c80-981e-da5756760ad0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:39.266340 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5975a6-eb8c-4f2f-b4b4-1d89e8beb133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:39.266938 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5975a6-eb8c-4f2f-b4b4-1d89e8beb133 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:39.267216 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5975a6-eb8c-4f2f-b4b4-1d89e8beb133 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:39.267489 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5975a6-eb8c-4f2f-b4b4-1d89e8beb133 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:39.268221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5975a6-eb8c-4f2f-b4b4-1d89e8beb133 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:39.268522 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5975a6-eb8c-4f2f-b4b4-1d89e8beb133 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:39.269932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5975a6-eb8c-4f2f-b4b4-1d89e8beb133 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:39.288069 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 409/806] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Sep 1 16:18:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:39.291730 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5975a6-eb8c-4f2f-b4b4-1d89e8beb133 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:39.343121 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:39.373912 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95623837-5056-4c80-981e-da5756760ad0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:39.375417 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:39.399109 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5975a6-eb8c-4f2f-b4b4-1d89e8beb133 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:39.408849 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeaeb7c-3920-43b0-9edf-01cfa5085a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:39.409477 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeaeb7c-3920-43b0-9edf-01cfa5085a22 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:39.409760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeaeb7c-3920-43b0-9edf-01cfa5085a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:39.410040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeaeb7c-3920-43b0-9edf-01cfa5085a22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:39.416790 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeaeb7c-3920-43b0-9edf-01cfa5085a22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:39.418541 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeaeb7c-3920-43b0-9edf-01cfa5085a22 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:39.429708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-8aeaeb7c-3920-43b0-9edf-01cfa5085a22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:39.429708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:39.429708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:39.429708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:39.429708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:39.429708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:39.429708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:39.429708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:39.429708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:39.429708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:39.429708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:39.429708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:39.429708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:39.429708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:39.429708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:39.430674 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:39.430674 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:39.430674 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:39.430674 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:39.430674 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:39.430674 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:39.430674 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:39.430674 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:39.430674 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:39.430674 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:39.430674 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:39.430674 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:39.430674 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:39.430674 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:39.430674 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:39.430674 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:39.430674 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:39.431605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:39.431605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:39.431605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:39.431605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:39.431605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:39.431605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:39.431605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:39.431605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:39.431605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 398/807] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:39.432112 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016200cb-1908-4d6e-a449-0695c5f3e1e6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:39.432283 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016200cb-1908-4d6e-a449-0695c5f3e1e6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:39.432569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016200cb-1908-4d6e-a449-0695c5f3e1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:39.436768 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc77ff-4410-4d88-a2a0-8c1bd8cd6d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:39.437372 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc77ff-4410-4d88-a2a0-8c1bd8cd6d69 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:39.437651 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc77ff-4410-4d88-a2a0-8c1bd8cd6d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:39.437923 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc77ff-4410-4d88-a2a0-8c1bd8cd6d69 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:39.438691 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc77ff-4410-4d88-a2a0-8c1bd8cd6d69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:39.438990 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc77ff-4410-4d88-a2a0-8c1bd8cd6d69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:39.440338 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc77ff-4410-4d88-a2a0-8c1bd8cd6d69 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:39.451031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-016200cb-1908-4d6e-a449-0695c5f3e1e6 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:39.452211 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-016200cb-1908-4d6e-a449-0695c5f3e1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:39.452351 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 410/808] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:39.453008 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc77ff-4410-4d88-a2a0-8c1bd8cd6d69 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:39.518850 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 399/809] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:18:39.519691 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016200cb-1908-4d6e-a449-0695c5f3e1e6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:39.556426 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc77ff-4410-4d88-a2a0-8c1bd8cd6d69 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:39.590616 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae3e57-5ad6-422b-9baf-d1849eef3ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:39.591221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae3e57-5ad6-422b-9baf-d1849eef3ccf None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:39.591475 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:39.591577 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae3e57-5ad6-422b-9baf-d1849eef3ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:39.591763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae3e57-5ad6-422b-9baf-d1849eef3ccf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:39.596807 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:39.636413 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016200cb-1908-4d6e-a449-0695c5f3e1e6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:39.674718 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dae3e57-5ad6-422b-9baf-d1849eef3ccf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:39.676286 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dae3e57-5ad6-422b-9baf-d1849eef3ccf None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:39.680763 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23f813-075c-41f5-8c12-ad39afca8591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:39.681396 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23f813-075c-41f5-8c12-ad39afca8591 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:39.681680 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23f813-075c-41f5-8c12-ad39afca8591 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:39.681954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23f813-075c-41f5-8c12-ad39afca8591 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:39.682571 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23f813-075c-41f5-8c12-ad39afca8591 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:39.683994 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23f813-075c-41f5-8c12-ad39afca8591 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:39.692953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-dc23f813-075c-41f5-8c12-ad39afca8591 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:18:39.692953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:39.692953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:18:39.692953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:18:39.692953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:18:39.692953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:18:39.692953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:18:39.692953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:39.692953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:18:39.692953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:39.692953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:18:39.692953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:39.692953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:39.692953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:39.692953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:39.692953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:39.692953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:39.693954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:39.693954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:39.693954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:39.693954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:39.693954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:39.693954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:39.693954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:18:39.693954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:18:39.693954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:18:39.693954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:18:39.693954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:39.693954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:39.693954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:39.693954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:39.693954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:39.693954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:39.695425 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:39.695425 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:39.695425 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:39.695425 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:39.695425 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:39.695425 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:39.695425 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:39.695425 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:39.695425 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:39.695425 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:39.695425 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:18:39.695425 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:39.695425 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:18:39.695425 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:39.695425 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 400/810] 10.222.0.58 () {56 vars in 1129 bytes} [Thu Sep 1 16:18:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:39.696239 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23f813-075c-41f5-8c12-ad39afca8591 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:39.793296 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 411/811] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:18:39 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:18:39.794144 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae3e57-5ad6-422b-9baf-d1849eef3ccf None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:39.807908 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23f813-075c-41f5-8c12-ad39afca8591 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:39.828708 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:39.832712 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:39.842584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d4fc3-44d4-421b-bf46-8ee79442e0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:39.843185 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d4fc3-44d4-421b-bf46-8ee79442e0d4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:39.843467 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d4fc3-44d4-421b-bf46-8ee79442e0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:39.843746 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d4fc3-44d4-421b-bf46-8ee79442e0d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:39.844447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d4fc3-44d4-421b-bf46-8ee79442e0d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:39.844765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d4fc3-44d4-421b-bf46-8ee79442e0d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:39.846206 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d4fc3-44d4-421b-bf46-8ee79442e0d4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:39.858571 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 401/812] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:39.859352 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d4fc3-44d4-421b-bf46-8ee79442e0d4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:39.894429 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae3e57-5ad6-422b-9baf-d1849eef3ccf None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:39.927799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a787cea-baa8-4007-acb7-0b5367deff4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:39.928412 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a787cea-baa8-4007-acb7-0b5367deff4e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:39.928687 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a787cea-baa8-4007-acb7-0b5367deff4e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:39.928960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a787cea-baa8-4007-acb7-0b5367deff4e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:39.935698 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a787cea-baa8-4007-acb7-0b5367deff4e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:39.937117 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a787cea-baa8-4007-acb7-0b5367deff4e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:39.948193 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-4a787cea-baa8-4007-acb7-0b5367deff4e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:18:39.948193 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:39.948193 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:39.948193 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:39.948193 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:39.948193 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:39.948193 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:39.948193 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:39.948193 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:39.948193 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:39.948193 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:39.948193 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:39.948193 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:39.948193 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:39.948193 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:39.948193 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:39.949153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:39.949153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:39.949153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:39.949153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:39.949153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:39.949153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:39.949153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:39.949153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:39.949153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:39.949153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:39.949153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:39.949153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:39.949153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:39.949153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:39.949153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:39.949153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:39.949153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:39.949153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:39.950136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:39.950136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:39.950136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:39.950136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:39.950136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:39.950136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:39.950136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:39.950136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:39.950136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:18:39.950136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:39.950136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 412/813] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:18:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:39.950136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a787cea-baa8-4007-acb7-0b5367deff4e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:39.961685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d4fc3-44d4-421b-bf46-8ee79442e0d4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:39.998675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1ae60-0ec7-44e6-9ee1-89a249103982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:39.999290 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1ae60-0ec7-44e6-9ee1-89a249103982 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:39.999581 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1ae60-0ec7-44e6-9ee1-89a249103982 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:40.000213 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1ae60-0ec7-44e6-9ee1-89a249103982 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:40.000862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1ae60-0ec7-44e6-9ee1-89a249103982 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:40.001482 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1ae60-0ec7-44e6-9ee1-89a249103982 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:40.002884 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1ae60-0ec7-44e6-9ee1-89a249103982 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:40.007941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 402/814] 10.222.0.58 () {56 vars in 1124 bytes} [Thu Sep 1 16:18:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:40.008722 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1ae60-0ec7-44e6-9ee1-89a249103982 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:40.058362 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:40.058737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:40.067699 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a787cea-baa8-4007-acb7-0b5367deff4e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:40.100406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcae6db-d3f9-44c1-9211-53a604a8294a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:40.101634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcae6db-d3f9-44c1-9211-53a604a8294a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:40.101784 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcae6db-d3f9-44c1-9211-53a604a8294a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:40.102044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcae6db-d3f9-44c1-9211-53a604a8294a None admin] PATH_INFO: `/v3/services/network` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:40.102655 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcae6db-d3f9-44c1-9211-53a604a8294a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:40.103989 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcae6db-d3f9-44c1-9211-53a604a8294a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:40.107751 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1ae60-0ec7-44e6-9ee1-89a249103982 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:40.112528 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-bdcae6db-d3f9-44c1-9211-53a604a8294a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:18:40.112528 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:40.112528 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:18:40.112528 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:18:40.112528 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:18:40.112528 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:18:40.112528 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:18:40.112528 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:40.112528 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:18:40.112528 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:40.112528 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:18:40.112528 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:40.112528 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:40.112528 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:40.112528 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:40.112528 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:40.112528 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:40.113692 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:40.113692 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:40.113692 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:40.113692 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:40.113692 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:40.113692 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:40.113692 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:18:40.113692 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:18:40.113692 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:18:40.113692 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:18:40.113692 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:40.113692 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:40.113692 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:40.113692 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:40.113692 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:40.113692 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:40.114629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:40.114629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:40.114629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:40.114629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:40.114629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:40.114629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:40.114629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:40.114629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:40.114629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:40.114629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:40.114629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:18:40.114629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:40.114629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:18:40.114629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:40.114629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 413/815] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:39 2022] GET /identity/v3/services/network => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:40.115437 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcae6db-d3f9-44c1-9211-53a604a8294a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:40.144881 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecff1eb-9054-4b03-b8f8-343e9b1bda64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:40.145440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecff1eb-9054-4b03-b8f8-343e9b1bda64 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:40.145717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecff1eb-9054-4b03-b8f8-343e9b1bda64 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:40.145991 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecff1eb-9054-4b03-b8f8-343e9b1bda64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:40.155637 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecff1eb-9054-4b03-b8f8-343e9b1bda64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:40.157104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecff1eb-9054-4b03-b8f8-343e9b1bda64 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:40.171713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-1ecff1eb-9054-4b03-b8f8-343e9b1bda64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:40.171713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:40.171713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:40.171713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:40.171713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:40.171713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:40.171713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:40.171713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:40.171713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:40.171713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:40.171713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:40.171713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:40.171713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:40.171713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:40.171713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:40.171713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:40.172717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:40.172717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:40.172717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:40.172717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:40.172717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:40.172717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:40.172717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:40.172717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:40.172717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:40.172717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:40.172717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:40.172717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:40.172717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:40.172717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:40.172717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:40.172717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:40.172717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:40.173643 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:40.173643 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:40.173643 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:40.173643 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 403/816] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:40.173643 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecff1eb-9054-4b03-b8f8-343e9b1bda64 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:40.213906 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcae6db-d3f9-44c1-9211-53a604a8294a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:40.247137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e9682-70b3-4110-8fbe-cb8813f071a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:40.247785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e9682-70b3-4110-8fbe-cb8813f071a1 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:40.248065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e9682-70b3-4110-8fbe-cb8813f071a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:40.248339 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e9682-70b3-4110-8fbe-cb8813f071a1 None admin] PATH_INFO: `/v3/projects/479530ced580426b85d1fc07af43417f/users/dd7b98536fa445cebd552fcbf54b757d/roles/b0923c7a240149ae9ece5b3c1622e860` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:40.262365 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82e9682-70b3-4110-8fbe-cb8813f071a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=479530ced580426b85d1fc07af43417f, user_id=dd7b98536fa445cebd552fcbf54b757d, role_id=b0923c7a240149ae9ece5b3c1622e860)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:40.263786 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82e9682-70b3-4110-8fbe-cb8813f071a1 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:40.275182 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecff1eb-9054-4b03-b8f8-343e9b1bda64 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:40.276712 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:40.277100 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 414/817] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Sep 1 16:18:40 2022] PUT /identity/v3/projects/479530ced580426b85d1fc07af43417f/users/dd7b98536fa445cebd552fcbf54b757d/roles/b0923c7a240149ae9ece5b3c1622e860 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:18:40.277968 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e9682-70b3-4110-8fbe-cb8813f071a1 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:40.300761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:40.328220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70769c1-df75-4b9c-a02c-b561afd68962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:40.328960 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70769c1-df75-4b9c-a02c-b561afd68962 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:40.329247 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70769c1-df75-4b9c-a02c-b561afd68962 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:40.329529 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70769c1-df75-4b9c-a02c-b561afd68962 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:40.330349 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70769c1-df75-4b9c-a02c-b561afd68962 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:40.334922 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70769c1-df75-4b9c-a02c-b561afd68962 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:40.336634 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70769c1-df75-4b9c-a02c-b561afd68962 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:40.359509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 404/818] 10.222.0.58 () {56 vars in 1135 bytes} [Thu Sep 1 16:18:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:40.360480 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70769c1-df75-4b9c-a02c-b561afd68962 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:40.420316 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e9682-70b3-4110-8fbe-cb8813f071a1 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:40.462528 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd452f-9725-40b2-a3ad-9667d6950519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:40.463234 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd452f-9725-40b2-a3ad-9667d6950519 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:40.463519 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd452f-9725-40b2-a3ad-9667d6950519 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:40.463796 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd452f-9725-40b2-a3ad-9667d6950519 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:40.464436 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd452f-9725-40b2-a3ad-9667d6950519 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:40.464743 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd452f-9725-40b2-a3ad-9667d6950519 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:40.466114 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd452f-9725-40b2-a3ad-9667d6950519 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:40.471405 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 415/819] 10.222.0.58 () {56 vars in 1124 bytes} [Thu Sep 1 16:18:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:40.472185 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd452f-9725-40b2-a3ad-9667d6950519 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:40.483499 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70769c1-df75-4b9c-a02c-b561afd68962 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:40.519245 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b7603-3b50-4b8c-903b-bf2bf19fadfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:40.519932 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b7603-3b50-4b8c-903b-bf2bf19fadfc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:40.520274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b7603-3b50-4b8c-903b-bf2bf19fadfc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:40.520611 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b7603-3b50-4b8c-903b-bf2bf19fadfc None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:40.521307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b7603-3b50-4b8c-903b-bf2bf19fadfc None admin] RBAC: Adding query filter params (name=network) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:40.521673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b7603-3b50-4b8c-903b-bf2bf19fadfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:40.523664 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b7603-3b50-4b8c-903b-bf2bf19fadfc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:40.528437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 405/820] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Sep 1 16:18:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:40.529106 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b7603-3b50-4b8c-903b-bf2bf19fadfc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:40.556758 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:40.557970 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:40.584736 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd452f-9725-40b2-a3ad-9667d6950519 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:40.622145 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee9ab5-09d7-4bd4-8664-5678e711c34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:40.622848 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee9ab5-09d7-4bd4-8664-5678e711c34b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:40.623190 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee9ab5-09d7-4bd4-8664-5678e711c34b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:40.623525 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee9ab5-09d7-4bd4-8664-5678e711c34b None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:40.624240 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee9ab5-09d7-4bd4-8664-5678e711c34b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:40.624541 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee9ab5-09d7-4bd4-8664-5678e711c34b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:40.625895 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee9ab5-09d7-4bd4-8664-5678e711c34b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:40.637098 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b7603-3b50-4b8c-903b-bf2bf19fadfc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:40.642217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 416/821] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:40.642217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee9ab5-09d7-4bd4-8664-5678e711c34b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:40.681253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a9552-6379-4c18-b839-1d3fadc9804a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:40.682155 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a9552-6379-4c18-b839-1d3fadc9804a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:40.682639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a9552-6379-4c18-b839-1d3fadc9804a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:40.683190 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a9552-6379-4c18-b839-1d3fadc9804a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:40.698719 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a9552-6379-4c18-b839-1d3fadc9804a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:40.700380 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a9552-6379-4c18-b839-1d3fadc9804a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:40.716585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-203a9552-6379-4c18-b839-1d3fadc9804a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:40.716585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:40.716585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:40.716585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:40.716585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:40.716585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:40.716585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:40.716585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:40.716585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:40.716585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:40.716585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:40.716585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:40.716585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:40.716585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:40.716585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:40.716585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:40.717606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:40.717606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:40.717606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:40.717606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:40.717606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:40.717606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:40.717606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:40.717606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:40.717606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:40.717606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:40.717606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:40.717606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:40.717606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:40.717606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:40.717606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:40.717606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:40.717606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:40.718565 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:40.718565 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:40.718565 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:40.722682 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 406/822] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:40.723654 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a9552-6379-4c18-b839-1d3fadc9804a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:40.754281 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee9ab5-09d7-4bd4-8664-5678e711c34b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:40.792991 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c10e8-8b84-4b0e-b389-bc498dc62862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:40.793606 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c10e8-8b84-4b0e-b389-bc498dc62862 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:40.793883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c10e8-8b84-4b0e-b389-bc498dc62862 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:40.794163 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c10e8-8b84-4b0e-b389-bc498dc62862 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:40.794841 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c10e8-8b84-4b0e-b389-bc498dc62862 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:40.796205 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c10e8-8b84-4b0e-b389-bc498dc62862 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:40.801224 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:40.811074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:40.822797 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 417/823] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:18:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:18:40.823912 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c10e8-8b84-4b0e-b389-bc498dc62862 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:40.855274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a9552-6379-4c18-b839-1d3fadc9804a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:40.908625 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e283cb-2d6e-4891-95d4-9e58799fe145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:40.909560 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e283cb-2d6e-4891-95d4-9e58799fe145 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:40.909985 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e283cb-2d6e-4891-95d4-9e58799fe145 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:40.910465 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e283cb-2d6e-4891-95d4-9e58799fe145 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:40.911316 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e283cb-2d6e-4891-95d4-9e58799fe145 None admin] RBAC: Adding query filter params (type=network) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:40.911879 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e283cb-2d6e-4891-95d4-9e58799fe145 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:40.913684 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e283cb-2d6e-4891-95d4-9e58799fe145 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:40.925133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 407/824] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Sep 1 16:18:40 2022] GET /identity/v3/services?type=network => generated 350 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:40.926364 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e283cb-2d6e-4891-95d4-9e58799fe145 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:40.955914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c10e8-8b84-4b0e-b389-bc498dc62862 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:41.001574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18ce320-aa6c-433b-97e3-7bfa3019c137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:41.002250 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ce320-aa6c-433b-97e3-7bfa3019c137 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:41.002533 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ce320-aa6c-433b-97e3-7bfa3019c137 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:41.002805 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ce320-aa6c-433b-97e3-7bfa3019c137 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:41.012070 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18ce320-aa6c-433b-97e3-7bfa3019c137 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:41.013514 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18ce320-aa6c-433b-97e3-7bfa3019c137 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:41.026034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-a18ce320-aa6c-433b-97e3-7bfa3019c137 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:41.026034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:41.026034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:41.026034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:41.026034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:41.026034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:41.026034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:41.026034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:41.026034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:41.026034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:41.026034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:41.026034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:41.026034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:41.026034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:41.026034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:41.027209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:41.027209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:41.027209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:41.027209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:41.027209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:41.027209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:41.027209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:41.027209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:41.027209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:41.027209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:41.027209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:41.027209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:41.027209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:41.027209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:41.027209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:41.027209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:41.027209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:41.028321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:41.028321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:41.028321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:41.028321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:41.028321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:41.028321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:41.028321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:41.028321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:41.029253 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 418/825] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:41.029895 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18ce320-aa6c-433b-97e3-7bfa3019c137 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:41.073332 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e283cb-2d6e-4891-95d4-9e58799fe145 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:41.105441 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:41.119570 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d2512-7e2c-47b8-b738-bd55d5c5113c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:41.120390 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d2512-7e2c-47b8-b738-bd55d5c5113c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:41.120734 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d2512-7e2c-47b8-b738-bd55d5c5113c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:41.121321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d2512-7e2c-47b8-b738-bd55d5c5113c None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:41.122267 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706d2512-7e2c-47b8-b738-bd55d5c5113c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:41.122592 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706d2512-7e2c-47b8-b738-bd55d5c5113c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:41.124145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706d2512-7e2c-47b8-b738-bd55d5c5113c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:41.139793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 408/826] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:41.141020 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d2512-7e2c-47b8-b738-bd55d5c5113c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:41.172860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18ce320-aa6c-433b-97e3-7bfa3019c137 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:41.182548 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:41.223963 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4045038-0f02-4da9-b472-8737dbc9c9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:41.225489 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4045038-0f02-4da9-b472-8737dbc9c9da None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:41.225894 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4045038-0f02-4da9-b472-8737dbc9c9da None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:41.226288 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4045038-0f02-4da9-b472-8737dbc9c9da None admin] PATH_INFO: `/v3/endpoints` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:41.228223 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4045038-0f02-4da9-b472-8737dbc9c9da None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:41.230168 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4045038-0f02-4da9-b472-8737dbc9c9da None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:41.258057 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d2512-7e2c-47b8-b738-bd55d5c5113c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:41.260239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 419/827] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:18:41 2022] POST /identity/v3/endpoints => generated 337 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:18:41.261305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4045038-0f02-4da9-b472-8737dbc9c9da None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:41.311423 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8ed6e-d35b-49c8-abaf-167d517aae2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:41.312399 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8ed6e-d35b-49c8-abaf-167d517aae2a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:41.312903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8ed6e-d35b-49c8-abaf-167d517aae2a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:41.313349 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8ed6e-d35b-49c8-abaf-167d517aae2a None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:41.314381 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b8ed6e-d35b-49c8-abaf-167d517aae2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:41.314878 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b8ed6e-d35b-49c8-abaf-167d517aae2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:41.317329 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b8ed6e-d35b-49c8-abaf-167d517aae2a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:41.333209 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 409/828] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:41.340006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8ed6e-d35b-49c8-abaf-167d517aae2a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:41.403541 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:41.476296 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8ed6e-d35b-49c8-abaf-167d517aae2a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:41.482471 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4045038-0f02-4da9-b472-8737dbc9c9da None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:41.493627 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:41.522671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f28cf6-24aa-44b5-aa7b-a2e7c8e221e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:41.523828 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f28cf6-24aa-44b5-aa7b-a2e7c8e221e6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:41.524497 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f28cf6-24aa-44b5-aa7b-a2e7c8e221e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:41.524996 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f28cf6-24aa-44b5-aa7b-a2e7c8e221e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:41.526363 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f28cf6-24aa-44b5-aa7b-a2e7c8e221e6 None admin] RBAC: Adding query filter params (role.id=ff293acb8551463aafe6970230042b89, scope.project.id=479530ced580426b85d1fc07af43417f, user.id=1f0c99fdf8bd4331ad7e641ee38554c1) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:41.526955 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f28cf6-24aa-44b5-aa7b-a2e7c8e221e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:41.528627 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f28cf6-24aa-44b5-aa7b-a2e7c8e221e6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:41.534025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce5f05-79b9-4323-b8f4-5f3b9488b7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:41.534714 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce5f05-79b9-4323-b8f4-5f3b9488b7a0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:41.534996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce5f05-79b9-4323-b8f4-5f3b9488b7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:41.535268 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce5f05-79b9-4323-b8f4-5f3b9488b7a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:41.535843 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 410/829] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:18:41 2022] GET /identity/v3/role_assignments?user.id=1f0c99fdf8bd4331ad7e641ee38554c1&scope.project.id=479530ced580426b85d1fc07af43417f&role.id=ff293acb8551463aafe6970230042b89 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:18:41.548867 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce5f05-79b9-4323-b8f4-5f3b9488b7a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:41.548867 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce5f05-79b9-4323-b8f4-5f3b9488b7a0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:41.562170 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-cfce5f05-79b9-4323-b8f4-5f3b9488b7a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:41.562170 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:41.562170 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:41.562170 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:41.562170 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:41.562170 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:41.562170 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:41.562170 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:41.562170 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:41.562170 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:41.562170 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:41.562170 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:41.562170 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:41.562170 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:41.562170 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:41.563155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:41.563155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:41.563155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:41.563155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:41.563155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:41.563155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:41.563155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:41.563155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:41.563155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:41.563155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:41.563155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:41.563155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:41.563155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:41.563155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:41.563155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:41.563155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:41.563155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:41.564085 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:41.564085 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:41.564085 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:41.564085 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:41.564085 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:41.564085 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:41.564085 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:41.564085 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:41.564085 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 420/830] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:41.569767 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f28cf6-24aa-44b5-aa7b-a2e7c8e221e6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:41.699212 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f28cf6-24aa-44b5-aa7b-a2e7c8e221e6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:41.717771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:41.741218 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960b533-cb3f-42b9-ad7d-bf7a284f138d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:41.741802 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960b533-cb3f-42b9-ad7d-bf7a284f138d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:41.742089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960b533-cb3f-42b9-ad7d-bf7a284f138d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:41.742392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960b533-cb3f-42b9-ad7d-bf7a284f138d None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:41.743094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4960b533-cb3f-42b9-ad7d-bf7a284f138d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:41.743404 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4960b533-cb3f-42b9-ad7d-bf7a284f138d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:41.744922 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4960b533-cb3f-42b9-ad7d-bf7a284f138d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:41.760055 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 411/831] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:41.765980 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce5f05-79b9-4323-b8f4-5f3b9488b7a0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:41.879180 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce5f05-79b9-4323-b8f4-5f3b9488b7a0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:41.915166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ef214-3505-47d1-a322-7e0316e4da61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:41.915822 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ef214-3505-47d1-a322-7e0316e4da61 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:41.916100 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ef214-3505-47d1-a322-7e0316e4da61 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:41.916372 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ef214-3505-47d1-a322-7e0316e4da61 None admin] PATH_INFO: `/v3/projects/479530ced580426b85d1fc07af43417f/users/e789695517c1437fb224f5a6195655f9/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:41.918629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ef214-3505-47d1-a322-7e0316e4da61 None admin] RBAC: Authorizing `identity:create_grant(project_id=479530ced580426b85d1fc07af43417f, user_id=e789695517c1437fb224f5a6195655f9, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:41.920236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ef214-3505-47d1-a322-7e0316e4da61 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:41.937006 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 421/832] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Sep 1 16:18:41 2022] PUT /identity/v3/projects/479530ced580426b85d1fc07af43417f/users/e789695517c1437fb224f5a6195655f9/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:18:42.296469 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1bce8-1b23-43a6-9973-9cd4c7d9284f None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:42.296859 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1bce8-1b23-43a6-9973-9cd4c7d9284f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:42.297053 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1bce8-1b23-43a6-9973-9cd4c7d9284f None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:42.298681 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 412/833] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:42.304510 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3122236-2496-4f60-a1b0-7de18685fcd7 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:42.304950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3122236-2496-4f60-a1b0-7de18685fcd7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:42.305045 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3122236-2496-4f60-a1b0-7de18685fcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:42.323171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-c3122236-2496-4f60-a1b0-7de18685fcd7 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:42.324333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c3122236-2496-4f60-a1b0-7de18685fcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:42.345001 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:42.398509 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 422/834] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:18:42.661308 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d1bce8-1b23-43a6-9973-9cd4c7d9284f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:42.771671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d1bce8-1b23-43a6-9973-9cd4c7d9284f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:42.808687 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829adba-19bc-4a36-acc1-5b26ef770353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:42.809602 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829adba-19bc-4a36-acc1-5b26ef770353 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:42.810060 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829adba-19bc-4a36-acc1-5b26ef770353 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:42.810526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829adba-19bc-4a36-acc1-5b26ef770353 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:42.822420 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f829adba-19bc-4a36-acc1-5b26ef770353 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:42.827590 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-f829adba-19bc-4a36-acc1-5b26ef770353 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:42.827590 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:42.827590 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:42.827590 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:42.827590 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:42.827590 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:42.827590 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:42.827590 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:42.827590 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:42.827590 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:42.827590 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:42.827590 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:42.827590 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:42.827590 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:42.827590 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:42.827590 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:42.828534 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:42.828534 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:42.828534 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:42.828534 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:42.828534 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:42.828534 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:42.828534 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:42.828534 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:42.828534 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:42.828534 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:42.828534 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:42.828534 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:42.828534 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:42.828534 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:42.828534 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:42.828534 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:42.829392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:42.829392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:42.829392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:42.829392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:42.829392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:42.829392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:42.829392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:42.829392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:42.829392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:42.829392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:42.829392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:42.829392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:42.831590 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 413/835] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Sep 1 16:18:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:42.837865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3122236-2496-4f60-a1b0-7de18685fcd7 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:42.940934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3122236-2496-4f60-a1b0-7de18685fcd7 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:42.974139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b1391-c1c0-49e9-9d1d-0ce012481e6c None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:42.974822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b1391-c1c0-49e9-9d1d-0ce012481e6c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:42.975365 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b1391-c1c0-49e9-9d1d-0ce012481e6c None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:42.977466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ae218-b1d4-4f06-b0be-5f6af18d9eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:42.977731 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 414/836] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:42.981339 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ae218-b1d4-4f06-b0be-5f6af18d9eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:42.981339 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ae218-b1d4-4f06-b0be-5f6af18d9eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:42.981339 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ae218-b1d4-4f06-b0be-5f6af18d9eb5 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:42.981339 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ae218-b1d4-4f06-b0be-5f6af18d9eb5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:42.981339 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ae218-b1d4-4f06-b0be-5f6af18d9eb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:42.981339 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ae218-b1d4-4f06-b0be-5f6af18d9eb5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:42.982770 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25308fdb-ca70-42a4-9e01-7361abb3473b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:42.983056 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25308fdb-ca70-42a4-9e01-7361abb3473b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:42.983332 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25308fdb-ca70-42a4-9e01-7361abb3473b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:42.999466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 423/837] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Sep 1 16:18:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:43.000955 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ae218-b1d4-4f06-b0be-5f6af18d9eb5 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:43.004168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-25308fdb-ca70-42a4-9e01-7361abb3473b None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:43.006398 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-25308fdb-ca70-42a4-9e01-7361abb3473b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:43.022497 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:43.045789 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:43.093528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 415/838] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:18:43.131939 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ae218-b1d4-4f06-b0be-5f6af18d9eb5 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:43.166808 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca58ad2b-00f3-45fb-8dae-fbecba70b169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:43.167979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58ad2b-00f3-45fb-8dae-fbecba70b169 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:43.168430 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58ad2b-00f3-45fb-8dae-fbecba70b169 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:43.168906 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58ad2b-00f3-45fb-8dae-fbecba70b169 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:43.180496 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca58ad2b-00f3-45fb-8dae-fbecba70b169 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:43.182133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca58ad2b-00f3-45fb-8dae-fbecba70b169 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:43.196697 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-ca58ad2b-00f3-45fb-8dae-fbecba70b169 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:43.196697 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:43.196697 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:43.196697 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:43.196697 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:43.196697 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:43.196697 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:43.196697 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:43.196697 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:43.196697 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:43.196697 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:43.196697 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:43.196697 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:43.196697 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:43.196697 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:43.196697 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:43.197652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:43.197652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:43.197652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:43.197652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:43.197652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:43.197652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:43.197652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:43.197652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:43.197652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:43.197652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:43.197652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:43.197652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:43.197652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:43.197652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:43.197652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:43.197652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:43.197652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:43.198595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:43.198595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:43.198595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:43.204811 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 424/839] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:18:43.206317 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25308fdb-ca70-42a4-9e01-7361abb3473b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:43.321360 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25308fdb-ca70-42a4-9e01-7361abb3473b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:43.355947 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45769eb-86ec-4b58-982f-5dca75407596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:43.356562 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45769eb-86ec-4b58-982f-5dca75407596 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:43.356844 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45769eb-86ec-4b58-982f-5dca75407596 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:43.357120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45769eb-86ec-4b58-982f-5dca75407596 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:43.357770 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45769eb-86ec-4b58-982f-5dca75407596 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:43.358076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45769eb-86ec-4b58-982f-5dca75407596 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:43.359518 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45769eb-86ec-4b58-982f-5dca75407596 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:43.360627 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca58ad2b-00f3-45fb-8dae-fbecba70b169 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:43.372786 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 416/840] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:43.378227 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45769eb-86ec-4b58-982f-5dca75407596 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:43.405629 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:43.436690 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:43.476557 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca58ad2b-00f3-45fb-8dae-fbecba70b169 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:43.495497 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45769eb-86ec-4b58-982f-5dca75407596 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:43.513996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f8ed9-923a-4791-a1ca-ac292cfcf9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:43.514669 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f8ed9-923a-4791-a1ca-ac292cfcf9de None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:43.514947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f8ed9-923a-4791-a1ca-ac292cfcf9de None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:43.515210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f8ed9-923a-4791-a1ca-ac292cfcf9de None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:43.515797 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99f8ed9-923a-4791-a1ca-ac292cfcf9de None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:43.517154 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99f8ed9-923a-4791-a1ca-ac292cfcf9de None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:43.526236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-a99f8ed9-923a-4791-a1ca-ac292cfcf9de None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:18:43.526236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:43.526236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:18:43.526236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:18:43.526236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:18:43.526236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:18:43.526236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:18:43.526236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:43.526236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:18:43.526236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:43.526236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:18:43.526236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:43.526236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:43.526236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:43.526236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:43.526236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:43.526236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:43.527256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:43.527256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:43.527256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:43.527256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:43.527256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:43.527256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:43.527256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:18:43.527256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:18:43.527256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:18:43.527256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:18:43.527256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:43.527256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:43.527256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:43.527256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:43.527256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:43.527256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:43.528146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:43.528146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:43.528146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:43.528146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:43.528146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:43.528146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:43.528146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:43.528146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:43.528146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:43.528146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:43.528146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:18:43.528146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:43.528146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:18:43.528146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:43.528146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 425/841] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Sep 1 16:18:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:43.535303 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f8ed9-923a-4791-a1ca-ac292cfcf9de None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:43.537739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165cc74-5c66-449e-b60f-b933f21981ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:43.538428 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165cc74-5c66-449e-b60f-b933f21981ff None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:43.538708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165cc74-5c66-449e-b60f-b933f21981ff None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:43.538978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165cc74-5c66-449e-b60f-b933f21981ff None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:43.549327 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165cc74-5c66-449e-b60f-b933f21981ff None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:43.551415 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165cc74-5c66-449e-b60f-b933f21981ff None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:43.564665 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-c165cc74-5c66-449e-b60f-b933f21981ff None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:18:43.564665 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:43.564665 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:43.564665 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:43.564665 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:43.564665 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:43.564665 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:43.564665 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:43.564665 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:43.564665 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:43.564665 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:43.564665 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:43.564665 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:43.564665 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:43.564665 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:43.564665 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:43.565642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:43.565642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:43.565642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:43.565642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:43.565642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:43.565642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:43.565642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:43.565642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:43.565642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:43.565642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:43.565642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:43.565642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:43.565642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:43.565642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:43.565642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:43.565642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:43.565642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:43.565642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:43.566630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:43.566630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:43.566630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:43.566630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:43.566630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:43.566630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:43.566630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:43.566630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:43.566630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:18:43.566630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:43.569172 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 417/842] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Sep 1 16:18:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:18:43.576872 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165cc74-5c66-449e-b60f-b933f21981ff None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:43.653781 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f8ed9-923a-4791-a1ca-ac292cfcf9de None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:43.676053 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:43.692583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67223477-e576-4b31-8125-a35fc95c223f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:43.693191 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67223477-e576-4b31-8125-a35fc95c223f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:43.693477 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67223477-e576-4b31-8125-a35fc95c223f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:43.693665 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165cc74-5c66-449e-b60f-b933f21981ff None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:43.693789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67223477-e576-4b31-8125-a35fc95c223f None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:43.694465 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67223477-e576-4b31-8125-a35fc95c223f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:43.694771 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67223477-e576-4b31-8125-a35fc95c223f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:43.696258 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67223477-e576-4b31-8125-a35fc95c223f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:43.697448 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:43.706387 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 426/843] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Sep 1 16:18:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:43.707960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5cbfbe-5510-4017-a4b8-58b49e1fd4d8 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:43.708254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5cbfbe-5510-4017-a4b8-58b49e1fd4d8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:43.708536 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5cbfbe-5510-4017-a4b8-58b49e1fd4d8 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:43.714339 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 427/844] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:43.714339 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5cbfbe-5510-4017-a4b8-58b49e1fd4d8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:43.734120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d60bd7-e1be-4f64-9dd4-dd319f0cea1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:43.734693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d60bd7-e1be-4f64-9dd4-dd319f0cea1b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:43.734975 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d60bd7-e1be-4f64-9dd4-dd319f0cea1b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:43.735249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d60bd7-e1be-4f64-9dd4-dd319f0cea1b None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:43.735978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d60bd7-e1be-4f64-9dd4-dd319f0cea1b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:43.736552 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d60bd7-e1be-4f64-9dd4-dd319f0cea1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:43.737693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d60bd7-e1be-4f64-9dd4-dd319f0cea1b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:43.758177 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 418/845] 10.222.0.58 () {56 vars in 1149 bytes} [Thu Sep 1 16:18:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:43.763942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9e128-8ec0-4f10-b405-5592cd7d64f7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:43.764300 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9e128-8ec0-4f10-b405-5592cd7d64f7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:43.764587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9e128-8ec0-4f10-b405-5592cd7d64f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:43.782862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-eeb9e128-8ec0-4f10-b405-5592cd7d64f7 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:43.784046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-eeb9e128-8ec0-4f10-b405-5592cd7d64f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:43.825079 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5cbfbe-5510-4017-a4b8-58b49e1fd4d8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:43.853552 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 419/846] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:18:43.854436 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9e128-8ec0-4f10-b405-5592cd7d64f7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:43.873277 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954508a-2350-465e-ba71-1cf596d6aebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:43.874228 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954508a-2350-465e-ba71-1cf596d6aebc None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:43.874711 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954508a-2350-465e-ba71-1cf596d6aebc None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:43.875233 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954508a-2350-465e-ba71-1cf596d6aebc None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:43.876598 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954508a-2350-465e-ba71-1cf596d6aebc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:43.877347 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954508a-2350-465e-ba71-1cf596d6aebc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:43.879664 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954508a-2350-465e-ba71-1cf596d6aebc None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:43.886904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 428/847] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Sep 1 16:18:43 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:43.943734 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:43.980861 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9e128-8ec0-4f10-b405-5592cd7d64f7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:44.002076 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85278fa1-5942-4c91-b509-3ae737065d7d None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:44.002395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85278fa1-5942-4c91-b509-3ae737065d7d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:44.002583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85278fa1-5942-4c91-b509-3ae737065d7d None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:44.004121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 429/848] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:18:44.009477 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873c799-1712-4fc1-a741-ed63dc36243b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:44.009786 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873c799-1712-4fc1-a741-ed63dc36243b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:44.010017 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873c799-1712-4fc1-a741-ed63dc36243b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:44.021270 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d6751-d432-4626-a0e5-d6aefac23036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:44.021895 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d6751-d432-4626-a0e5-d6aefac23036 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:44.022173 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d6751-d432-4626-a0e5-d6aefac23036 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:44.022528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d6751-d432-4626-a0e5-d6aefac23036 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:44.028798 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-0873c799-1712-4fc1-a741-ed63dc36243b None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:44.029980 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0873c799-1712-4fc1-a741-ed63dc36243b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:44.112689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:44.134546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 430/849] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:18:44.143598 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873c799-1712-4fc1-a741-ed63dc36243b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:44.215766 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d6751-d432-4626-a0e5-d6aefac23036 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:44.217211 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d6751-d432-4626-a0e5-d6aefac23036 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:44.267872 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873c799-1712-4fc1-a741-ed63dc36243b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:44.300016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:44.311785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ebd0d-c42e-4adf-9fc1-974b79a1e66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:44.312422 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ebd0d-c42e-4adf-9fc1-974b79a1e66a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:44.312979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ebd0d-c42e-4adf-9fc1-974b79a1e66a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:44.313252 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ebd0d-c42e-4adf-9fc1-974b79a1e66a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:44.313839 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ebd0d-c42e-4adf-9fc1-974b79a1e66a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:44.315659 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ebd0d-c42e-4adf-9fc1-974b79a1e66a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:44.327695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-562ebd0d-c42e-4adf-9fc1-974b79a1e66a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:18:44.327695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:44.327695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:18:44.327695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:18:44.327695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:18:44.327695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:18:44.327695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:18:44.327695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:44.327695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:18:44.327695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:44.327695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:18:44.327695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:44.327695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:44.327695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:44.327695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:44.327695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:44.328772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:44.328772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:44.328772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:44.328772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:44.328772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:44.328772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:44.328772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:44.328772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:18:44.328772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:18:44.328772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:18:44.328772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:18:44.328772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:44.328772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:44.328772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:44.328772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:44.328772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:44.328772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:44.329977 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:44.329977 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:44.329977 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:44.329977 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:44.329977 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:44.329977 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:44.329977 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:44.329977 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:44.329977 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:44.329977 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:44.329977 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:18:44.329977 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:44.329977 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:18:44.329977 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:44.329977 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 431/850] 10.222.0.58 () {56 vars in 1132 bytes} [Thu Sep 1 16:18:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:44.334080 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ebd0d-c42e-4adf-9fc1-974b79a1e66a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:44.334908 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 420/851] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:18:43 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:18:44.339488 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d6751-d432-4626-a0e5-d6aefac23036 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:44.411046 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:44.447765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d6751-d432-4626-a0e5-d6aefac23036 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:44.456938 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ebd0d-c42e-4adf-9fc1-974b79a1e66a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:44.483675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737d193-d2f2-4231-ae84-1df11e5b9bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:44.484282 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737d193-d2f2-4231-ae84-1df11e5b9bdd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:44.484565 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737d193-d2f2-4231-ae84-1df11e5b9bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:44.484838 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737d193-d2f2-4231-ae84-1df11e5b9bdd None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:44.485472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0737d193-d2f2-4231-ae84-1df11e5b9bdd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:44.485784 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0737d193-d2f2-4231-ae84-1df11e5b9bdd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:44.487170 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0737d193-d2f2-4231-ae84-1df11e5b9bdd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:44.492128 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 421/852] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Sep 1 16:18:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:44.492901 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737d193-d2f2-4231-ae84-1df11e5b9bdd None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:44.498431 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3cc8c-d125-46f7-8c7d-95c2fc7f6a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:44.499286 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3cc8c-d125-46f7-8c7d-95c2fc7f6a73 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:44.499746 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3cc8c-d125-46f7-8c7d-95c2fc7f6a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:44.500211 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3cc8c-d125-46f7-8c7d-95c2fc7f6a73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:44.512412 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3cc8c-d125-46f7-8c7d-95c2fc7f6a73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:44.514045 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3cc8c-d125-46f7-8c7d-95c2fc7f6a73 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:44.529550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-cdc3cc8c-d125-46f7-8c7d-95c2fc7f6a73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:44.529550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:44.529550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:44.529550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:44.529550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:44.529550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:44.529550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:44.529550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:44.529550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:44.529550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:44.529550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:44.529550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:44.529550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:44.529550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:44.529550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:44.529550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:44.530546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:44.530546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:44.530546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:44.530546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:44.530546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:44.530546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:44.530546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:44.530546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:44.530546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:44.530546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:44.530546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:44.530546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:44.530546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:44.530546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:44.530546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:44.530546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:44.530546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:44.531491 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:44.531491 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:44.531491 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:44.531491 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 432/853] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:18:44.532191 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3cc8c-d125-46f7-8c7d-95c2fc7f6a73 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:44.533489 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:44.602164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737d193-d2f2-4231-ae84-1df11e5b9bdd None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:44.640751 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72984b3-da79-4af6-b8f6-7dedd3e58e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:44.641505 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72984b3-da79-4af6-b8f6-7dedd3e58e57 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:44.641793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72984b3-da79-4af6-b8f6-7dedd3e58e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:44.642069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72984b3-da79-4af6-b8f6-7dedd3e58e57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:44.646974 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3cc8c-d125-46f7-8c7d-95c2fc7f6a73 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:44.653447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72984b3-da79-4af6-b8f6-7dedd3e58e57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:44.657356 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:44.658224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-e72984b3-da79-4af6-b8f6-7dedd3e58e57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:18:44.658224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:44.658224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:44.658224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:44.658224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:44.658224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:44.658224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:44.658224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:44.658224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:44.658224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:44.658224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:44.658224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:44.658224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:44.658224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:44.658224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:44.658224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:44.659159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:44.659159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:44.659159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:44.659159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:44.659159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:44.659159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:44.659159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:44.659159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:44.659159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:44.659159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:44.659159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:44.659159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:44.659159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:44.659159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:44.659159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:44.659159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:44.660041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:44.660041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:44.660041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:44.660041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:44.660041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:44.660041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:44.660041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:44.660041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:44.660041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:44.660041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:44.660041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:18:44.660041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:44.660041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 422/854] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:18:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:44.661572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72984b3-da79-4af6-b8f6-7dedd3e58e57 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:44.686140 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ac782-7929-4a62-8323-04503f766a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:44.686795 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ac782-7929-4a62-8323-04503f766a4a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:44.687073 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ac782-7929-4a62-8323-04503f766a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:44.687343 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ac782-7929-4a62-8323-04503f766a4a None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:44.687979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ac782-7929-4a62-8323-04503f766a4a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:44.688274 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ac782-7929-4a62-8323-04503f766a4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:44.689619 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ac782-7929-4a62-8323-04503f766a4a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:44.702223 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 433/855] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Sep 1 16:18:44 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:44.702223 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ac782-7929-4a62-8323-04503f766a4a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:44.793490 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:44.798586 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72984b3-da79-4af6-b8f6-7dedd3e58e57 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:44.825989 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ac782-7929-4a62-8323-04503f766a4a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:44.836062 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f1f84-bec7-4dc8-af3b-86f4a17c31f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:44.836846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f1f84-bec7-4dc8-af3b-86f4a17c31f5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:44.837225 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f1f84-bec7-4dc8-af3b-86f4a17c31f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:44.837592 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f1f84-bec7-4dc8-af3b-86f4a17c31f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:44.838398 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f1f84-bec7-4dc8-af3b-86f4a17c31f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:44.838807 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f1f84-bec7-4dc8-af3b-86f4a17c31f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:44.840299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f1f84-bec7-4dc8-af3b-86f4a17c31f5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:44.858738 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 423/856] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:44.858738 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f1f84-bec7-4dc8-af3b-86f4a17c31f5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:44.864987 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac67fcd-39e1-4ebf-a2d6-fd55d8be462a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:44.865615 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac67fcd-39e1-4ebf-a2d6-fd55d8be462a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:44.865879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac67fcd-39e1-4ebf-a2d6-fd55d8be462a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:44.866689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac67fcd-39e1-4ebf-a2d6-fd55d8be462a None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:44.867375 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac67fcd-39e1-4ebf-a2d6-fd55d8be462a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:44.867681 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac67fcd-39e1-4ebf-a2d6-fd55d8be462a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:44.869049 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac67fcd-39e1-4ebf-a2d6-fd55d8be462a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:44.879373 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 434/857] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:18:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:44.886597 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac67fcd-39e1-4ebf-a2d6-fd55d8be462a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:44.936074 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:44.980509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f1f84-bec7-4dc8-af3b-86f4a17c31f5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:45.017074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf5ea3-8a71-47fc-83bf-241ab77c2ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:45.017706 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf5ea3-8a71-47fc-83bf-241ab77c2ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:45.017988 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf5ea3-8a71-47fc-83bf-241ab77c2ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:45.018285 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf5ea3-8a71-47fc-83bf-241ab77c2ca4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:45.025465 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf5ea3-8a71-47fc-83bf-241ab77c2ca4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:45.026316 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac67fcd-39e1-4ebf-a2d6-fd55d8be462a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:45.027062 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf5ea3-8a71-47fc-83bf-241ab77c2ca4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:45.039903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-ddcf5ea3-8a71-47fc-83bf-241ab77c2ca4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:45.039903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:45.039903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:45.039903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:45.039903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:45.039903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:45.039903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:45.039903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:45.039903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:45.039903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:45.039903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:45.039903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:45.039903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:45.039903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:45.039903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:45.041071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:45.041071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:45.041071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:45.041071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:45.041071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:45.041071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:45.041071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:45.041071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:45.041071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:45.041071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:45.041071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:45.041071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:45.041071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:45.041071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:45.041071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:45.041071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:45.041071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:45.042286 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:45.042286 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:45.042286 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:45.042286 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:45.042286 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:45.042286 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:45.042286 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:45.042286 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:45.046260 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 424/858] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:45.047109 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf5ea3-8a71-47fc-83bf-241ab77c2ca4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:45.067477 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:45.073050 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166d448-717b-4de6-a330-894d39cfa125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:45.074033 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166d448-717b-4de6-a330-894d39cfa125 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:45.074596 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166d448-717b-4de6-a330-894d39cfa125 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:45.075071 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166d448-717b-4de6-a330-894d39cfa125 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:45.087994 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c166d448-717b-4de6-a330-894d39cfa125 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:45.089758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c166d448-717b-4de6-a330-894d39cfa125 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:45.105761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-c166d448-717b-4de6-a330-894d39cfa125 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:45.105761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:45.105761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:45.105761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:45.105761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:45.105761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:45.105761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:45.105761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:45.105761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:45.105761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:45.105761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:45.105761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:45.105761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:45.105761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:45.105761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:45.105761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:45.106781 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:45.106781 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:45.106781 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:45.106781 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:45.106781 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:45.106781 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:45.106781 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:45.106781 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:45.106781 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:45.106781 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:45.106781 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:45.106781 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:45.106781 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:45.106781 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:45.106781 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:45.106781 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:45.106781 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:45.107748 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:45.107748 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:45.107748 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:45.114493 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 435/859] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:18:45.115654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166d448-717b-4de6-a330-894d39cfa125 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:45.151988 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf5ea3-8a71-47fc-83bf-241ab77c2ca4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:45.186476 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021dfcef-a7d0-4fd2-a95d-f3647380a4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:45.187079 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021dfcef-a7d0-4fd2-a95d-f3647380a4af None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:45.187414 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021dfcef-a7d0-4fd2-a95d-f3647380a4af None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:45.187695 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021dfcef-a7d0-4fd2-a95d-f3647380a4af None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:45.188390 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021dfcef-a7d0-4fd2-a95d-f3647380a4af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:45.188686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021dfcef-a7d0-4fd2-a95d-f3647380a4af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:45.190459 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021dfcef-a7d0-4fd2-a95d-f3647380a4af None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:45.207714 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 425/860] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:45.209001 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021dfcef-a7d0-4fd2-a95d-f3647380a4af None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:45.212160 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:45.230278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166d448-717b-4de6-a330-894d39cfa125 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:45.268350 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ead6b-de3a-4ce6-ac77-ab5a96fed0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:45.269277 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ead6b-de3a-4ce6-ac77-ab5a96fed0e1 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:45.269749 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ead6b-de3a-4ce6-ac77-ab5a96fed0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:45.270221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ead6b-de3a-4ce6-ac77-ab5a96fed0e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:45.271142 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ead6b-de3a-4ce6-ac77-ab5a96fed0e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:45.271718 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ead6b-de3a-4ce6-ac77-ab5a96fed0e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:45.273787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ead6b-de3a-4ce6-ac77-ab5a96fed0e1 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:45.292752 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 436/861] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:45.294366 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ead6b-de3a-4ce6-ac77-ab5a96fed0e1 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:45.296691 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:45.320011 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021dfcef-a7d0-4fd2-a95d-f3647380a4af None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:45.354844 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f76f66-1790-4479-b03d-7c825153e03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:45.355540 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f76f66-1790-4479-b03d-7c825153e03c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:45.355870 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f76f66-1790-4479-b03d-7c825153e03c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:45.356217 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f76f66-1790-4479-b03d-7c825153e03c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:45.357012 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f76f66-1790-4479-b03d-7c825153e03c None admin] RBAC: Adding query filter params (role.id=b0923c7a240149ae9ece5b3c1622e860, scope.project.id=479530ced580426b85d1fc07af43417f, user.id=dd7b98536fa445cebd552fcbf54b757d) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:45.357391 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f76f66-1790-4479-b03d-7c825153e03c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:45.358977 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f76f66-1790-4479-b03d-7c825153e03c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:45.365126 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 426/862] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:18:45 2022] GET /identity/v3/role_assignments?user.id=dd7b98536fa445cebd552fcbf54b757d&scope.project.id=479530ced580426b85d1fc07af43417f&role.id=b0923c7a240149ae9ece5b3c1622e860 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:45.416390 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ead6b-de3a-4ce6-ac77-ab5a96fed0e1 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:45.456230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a9558-344a-4a04-a14d-191d2e6e759b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:45.456871 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a9558-344a-4a04-a14d-191d2e6e759b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:45.457150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a9558-344a-4a04-a14d-191d2e6e759b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:45.457421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a9558-344a-4a04-a14d-191d2e6e759b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:45.466315 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a9558-344a-4a04-a14d-191d2e6e759b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:45.467787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a9558-344a-4a04-a14d-191d2e6e759b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:45.481559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-5b7a9558-344a-4a04-a14d-191d2e6e759b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:18:45.481559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:45.481559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:45.481559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:45.481559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:45.481559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:45.481559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:45.481559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:45.481559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:45.481559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:45.481559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:45.481559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:45.481559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:45.481559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:45.481559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:45.481559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:45.483901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:45.483901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:45.483901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:45.483901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:45.483901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:45.483901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:45.483901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:45.483901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:45.483901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:45.483901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:45.483901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:45.483901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:45.483901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:45.483901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:45.483901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:45.483901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:45.483901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:45.483901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:45.484889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:45.484889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:45.484889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:45.484889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:45.484889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:45.484889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:45.484889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:45.484889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:45.484889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:18:45.484889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:45.484889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 437/863] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:18:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:45.492715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f76f66-1790-4479-b03d-7c825153e03c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:45.601490 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f76f66-1790-4479-b03d-7c825153e03c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:45.637307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c7991-0b6a-43dd-9f27-fcbcdd815c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:45.637919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c7991-0b6a-43dd-9f27-fcbcdd815c75 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:45.638222 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c7991-0b6a-43dd-9f27-fcbcdd815c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:45.638497 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c7991-0b6a-43dd-9f27-fcbcdd815c75 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:45.639226 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c7991-0b6a-43dd-9f27-fcbcdd815c75 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:45.639533 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c7991-0b6a-43dd-9f27-fcbcdd815c75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:45.640962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c7991-0b6a-43dd-9f27-fcbcdd815c75 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:45.656448 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:45.660605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 427/864] 10.222.0.58 () {56 vars in 1135 bytes} [Thu Sep 1 16:18:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:45.667266 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a9558-344a-4a04-a14d-191d2e6e759b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:45.677195 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:45.775928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a9558-344a-4a04-a14d-191d2e6e759b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:45.812403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61f4d2-c0d8-4268-aa70-379609f29464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:45.813374 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61f4d2-c0d8-4268-aa70-379609f29464 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:45.813877 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61f4d2-c0d8-4268-aa70-379609f29464 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:45.814387 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61f4d2-c0d8-4268-aa70-379609f29464 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:45.888747 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc80a5-478f-4b42-a14e-a239d832afac None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:45.889257 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc80a5-478f-4b42-a14e-a239d832afac None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:45.889257 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc80a5-478f-4b42-a14e-a239d832afac None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:45.890523 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61f4d2-c0d8-4268-aa70-379609f29464 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:45.890765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 428/865] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:45.892363 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61f4d2-c0d8-4268-aa70-379609f29464 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:45.895760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98827e4-a9c0-45b3-8ada-4a93f554b304 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:45.896054 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98827e4-a9c0-45b3-8ada-4a93f554b304 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:45.896335 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98827e4-a9c0-45b3-8ada-4a93f554b304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:45.900776 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:45.919517 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e98827e4-a9c0-45b3-8ada-4a93f554b304 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:45.921197 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e98827e4-a9c0-45b3-8ada-4a93f554b304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:46.002679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 429/866] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:18:46.017303 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 438/867] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:18:45 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:18:46.022919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98827e4-a9c0-45b3-8ada-4a93f554b304 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:46.127856 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98827e4-a9c0-45b3-8ada-4a93f554b304 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:46.157340 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:46.164018 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887dc15b-7429-4e3f-ab14-169bf8dbee0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:46.164654 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887dc15b-7429-4e3f-ab14-169bf8dbee0e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:46.165353 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887dc15b-7429-4e3f-ab14-169bf8dbee0e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:46.165632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887dc15b-7429-4e3f-ab14-169bf8dbee0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:46.176334 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887dc15b-7429-4e3f-ab14-169bf8dbee0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:46.177818 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887dc15b-7429-4e3f-ab14-169bf8dbee0e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:46.192718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-887dc15b-7429-4e3f-ab14-169bf8dbee0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:46.192718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:46.192718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:46.192718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:46.192718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:46.192718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:46.192718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:46.192718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:46.192718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:46.192718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:46.192718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:46.192718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:46.192718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:46.192718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:46.192718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:46.192718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:46.193672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:46.193672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:46.193672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:46.193672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:46.193672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:46.193672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:46.193672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:46.193672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:46.193672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:46.193672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:46.193672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:46.193672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:46.193672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:46.193672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:46.193672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:46.193672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:46.193672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:46.194641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:46.194641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:46.194641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:46.194641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 430/868] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:46.199730 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61f4d2-c0d8-4268-aa70-379609f29464 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:46.294154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887dc15b-7429-4e3f-ab14-169bf8dbee0e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:46.300932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61f4d2-c0d8-4268-aa70-379609f29464 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:46.310498 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:46.345359 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d801796-71ce-4800-b834-9ef5867079fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:46.345969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d801796-71ce-4800-b834-9ef5867079fc None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:46.346272 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d801796-71ce-4800-b834-9ef5867079fc None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:46.346549 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d801796-71ce-4800-b834-9ef5867079fc None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:46.347191 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d801796-71ce-4800-b834-9ef5867079fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:46.347491 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d801796-71ce-4800-b834-9ef5867079fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:46.349380 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d801796-71ce-4800-b834-9ef5867079fc None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:46.364108 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 439/869] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:46.369991 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d801796-71ce-4800-b834-9ef5867079fc None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:46.412406 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887dc15b-7429-4e3f-ab14-169bf8dbee0e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:46.451820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb0cc7-c87d-493d-8850-7f8b52eca127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:46.453061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb0cc7-c87d-493d-8850-7f8b52eca127 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:46.454250 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb0cc7-c87d-493d-8850-7f8b52eca127 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:46.454741 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb0cc7-c87d-493d-8850-7f8b52eca127 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:46.455866 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb0cc7-c87d-493d-8850-7f8b52eca127 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:46.458689 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb0cc7-c87d-493d-8850-7f8b52eca127 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:46.485521 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 431/870] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Sep 1 16:18:46 2022] POST /identity/v3/services => generated 251 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:18:46.500772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d801796-71ce-4800-b834-9ef5867079fc None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:46.556437 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ea869-787c-4d38-98e4-eba0367d704e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:46.557420 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ea869-787c-4d38-98e4-eba0367d704e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:46.557881 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ea869-787c-4d38-98e4-eba0367d704e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:46.558380 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ea869-787c-4d38-98e4-eba0367d704e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:46.566894 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ea869-787c-4d38-98e4-eba0367d704e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:46.568674 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ea869-787c-4d38-98e4-eba0367d704e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:46.581877 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-4b9ea869-787c-4d38-98e4-eba0367d704e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:46.581877 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:46.581877 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:46.581877 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:46.581877 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:46.581877 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:46.581877 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:46.581877 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:46.581877 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:46.581877 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:46.581877 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:46.581877 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:46.581877 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:46.581877 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:46.581877 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:46.582907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:46.582907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:46.582907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:46.582907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:46.582907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:46.582907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:46.582907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:46.582907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:46.582907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:46.582907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:46.582907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:46.582907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:46.582907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:46.582907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:46.582907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:46.582907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:46.582907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:46.583817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:46.583817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:46.583817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:46.583817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:46.583817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:46.583817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:46.583817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:46.583817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:46.586015 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 440/871] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:18:46.592177 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb0cc7-c87d-493d-8850-7f8b52eca127 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:46.643629 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:46.710594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98f4f2-097c-4051-a865-d938cde762f7 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:46.710892 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98f4f2-097c-4051-a865-d938cde762f7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:46.710996 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb0cc7-c87d-493d-8850-7f8b52eca127 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:46.711076 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98f4f2-097c-4051-a865-d938cde762f7 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:46.712392 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 441/872] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:18:46.717504 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacb2e3-3ccd-4d44-b253-3da6a4541482 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:46.717787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacb2e3-3ccd-4d44-b253-3da6a4541482 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:46.718068 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacb2e3-3ccd-4d44-b253-3da6a4541482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:46.736788 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-acacb2e3-3ccd-4d44-b253-3da6a4541482 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:46.737980 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-acacb2e3-3ccd-4d44-b253-3da6a4541482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:46.753511 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5038ff-e586-4d1a-8a0c-758bcf35da63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:46.754283 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5038ff-e586-4d1a-8a0c-758bcf35da63 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:46.754574 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5038ff-e586-4d1a-8a0c-758bcf35da63 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:46.754848 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5038ff-e586-4d1a-8a0c-758bcf35da63 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:46.755711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5038ff-e586-4d1a-8a0c-758bcf35da63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:46.756019 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5038ff-e586-4d1a-8a0c-758bcf35da63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:46.757485 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:46.758299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5038ff-e586-4d1a-8a0c-758bcf35da63 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:46.775972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 432/873] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:46.781888 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5038ff-e586-4d1a-8a0c-758bcf35da63 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:46.830304 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 442/874] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:46 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:18:46.895808 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5038ff-e586-4d1a-8a0c-758bcf35da63 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:46.899046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:46.935782 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f13678-e225-4030-980e-5070a95eb49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:46.936616 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f13678-e225-4030-980e-5070a95eb49f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:46.937071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f13678-e225-4030-980e-5070a95eb49f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:46.937493 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f13678-e225-4030-980e-5070a95eb49f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:46.938416 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f13678-e225-4030-980e-5070a95eb49f None admin] RBAC: Adding query filter params (role.id=ff293acb8551463aafe6970230042b89, scope.project.id=479530ced580426b85d1fc07af43417f, user.id=e789695517c1437fb224f5a6195655f9) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:46.938871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f13678-e225-4030-980e-5070a95eb49f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:46.940433 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f13678-e225-4030-980e-5070a95eb49f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:46.947045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 433/875] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:18:46 2022] GET /identity/v3/role_assignments?user.id=e789695517c1437fb224f5a6195655f9&scope.project.id=479530ced580426b85d1fc07af43417f&role.id=ff293acb8551463aafe6970230042b89 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:47.097222 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacb2e3-3ccd-4d44-b253-3da6a4541482 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:47.204997 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacb2e3-3ccd-4d44-b253-3da6a4541482 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:47.244557 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d25911-4345-405c-8510-4711b9190278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:47.245476 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d25911-4345-405c-8510-4711b9190278 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:47.245934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d25911-4345-405c-8510-4711b9190278 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:47.246379 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d25911-4345-405c-8510-4711b9190278 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:47.247175 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d25911-4345-405c-8510-4711b9190278 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:47.248676 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d25911-4345-405c-8510-4711b9190278 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:47.270328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 443/876] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Sep 1 16:18:47 2022] POST /identity/v3/services => generated 269 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:18:47.509634 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba539c-236e-419b-80d3-c99636a66140 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:47.509918 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba539c-236e-419b-80d3-c99636a66140 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:47.510219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba539c-236e-419b-80d3-c99636a66140 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:47.511812 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 434/877] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:47.516953 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aea9f0-d4d5-47f2-81c6-cfd0b9d99538 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:47.517437 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aea9f0-d4d5-47f2-81c6-cfd0b9d99538 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:47.517620 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aea9f0-d4d5-47f2-81c6-cfd0b9d99538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:47.536407 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-41aea9f0-d4d5-47f2-81c6-cfd0b9d99538 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:47.537634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-41aea9f0-d4d5-47f2-81c6-cfd0b9d99538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:47.546671 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:47.618577 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 444/878] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:18:47.887383 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba539c-236e-419b-80d3-c99636a66140 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:47.995935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba539c-236e-419b-80d3-c99636a66140 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:48.031835 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658719fc-527f-40d4-94bd-25f9d3d88cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:48.032315 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658719fc-527f-40d4-94bd-25f9d3d88cdc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:48.032600 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658719fc-527f-40d4-94bd-25f9d3d88cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:48.032876 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658719fc-527f-40d4-94bd-25f9d3d88cdc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:48.049533 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658719fc-527f-40d4-94bd-25f9d3d88cdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:48.054976 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-658719fc-527f-40d4-94bd-25f9d3d88cdc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:18:48.054976 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:48.054976 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:48.054976 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:48.054976 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:48.054976 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:48.054976 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:48.054976 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:48.054976 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:48.054976 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:48.054976 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:48.054976 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:48.054976 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:48.054976 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:48.054976 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:48.054976 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:48.055935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:48.055935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:48.055935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:48.055935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:48.055935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:48.055935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:48.055935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:48.055935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:48.055935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:48.055935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:48.055935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:48.055935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:48.055935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:48.055935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:48.055935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:48.055935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:48.056816 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:48.056816 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:48.056816 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:48.056816 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:48.056816 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:48.056816 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:48.056816 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:48.056816 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:48.056816 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:48.056816 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:48.056816 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:18:48.056816 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:48.056816 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 435/879] 10.222.0.58 () {56 vars in 1120 bytes} [Thu Sep 1 16:18:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:48.064123 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aea9f0-d4d5-47f2-81c6-cfd0b9d99538 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:48.170764 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aea9f0-d4d5-47f2-81c6-cfd0b9d99538 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:48.206446 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45322c-aa68-4bc1-be38-eccdbe6fb8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:48.207044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45322c-aa68-4bc1-be38-eccdbe6fb8a8 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:48.207317 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45322c-aa68-4bc1-be38-eccdbe6fb8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:48.207581 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45322c-aa68-4bc1-be38-eccdbe6fb8a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:48.208239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45322c-aa68-4bc1-be38-eccdbe6fb8a8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:48.208532 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45322c-aa68-4bc1-be38-eccdbe6fb8a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:48.210419 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45322c-aa68-4bc1-be38-eccdbe6fb8a8 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:48.215478 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:48.219116 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 445/880] 10.222.0.58 () {56 vars in 1115 bytes} [Thu Sep 1 16:18:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:48.225071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658719fc-527f-40d4-94bd-25f9d3d88cdc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:48.257238 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:48.338516 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658719fc-527f-40d4-94bd-25f9d3d88cdc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:48.374429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93917d4d-08fd-4c7c-b042-85da6f166b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:48.375385 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93917d4d-08fd-4c7c-b042-85da6f166b23 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:48.375945 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93917d4d-08fd-4c7c-b042-85da6f166b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:48.376418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93917d4d-08fd-4c7c-b042-85da6f166b23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:48.387005 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93917d4d-08fd-4c7c-b042-85da6f166b23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:48.388802 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93917d4d-08fd-4c7c-b042-85da6f166b23 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:48.405767 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-93917d4d-08fd-4c7c-b042-85da6f166b23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:48.405767 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:48.405767 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:48.405767 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:48.405767 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:48.405767 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:48.405767 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:48.405767 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:48.405767 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:48.405767 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:48.405767 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:48.405767 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:48.405767 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:48.405767 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:48.405767 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:48.405767 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:48.406840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:48.406840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:48.406840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:48.406840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:48.406840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:48.406840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:48.406840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:48.406840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:48.406840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:48.406840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:48.406840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:48.406840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:48.406840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:48.406840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:48.406840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:48.406840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:48.406840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:48.407802 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:48.407802 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:48.407802 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:48.409966 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 436/881] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:48.416553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45322c-aa68-4bc1-be38-eccdbe6fb8a8 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:48.523361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45322c-aa68-4bc1-be38-eccdbe6fb8a8 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:48.559487 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252958e-d69b-437e-af9c-0fa30c581a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:48.560484 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252958e-d69b-437e-af9c-0fa30c581a01 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:48.560963 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252958e-d69b-437e-af9c-0fa30c581a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:48.561503 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252958e-d69b-437e-af9c-0fa30c581a01 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:48.562396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e252958e-d69b-437e-af9c-0fa30c581a01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:48.562899 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e252958e-d69b-437e-af9c-0fa30c581a01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:48.564528 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e252958e-d69b-437e-af9c-0fa30c581a01 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:48.581437 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 446/882] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:48.587594 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93917d4d-08fd-4c7c-b042-85da6f166b23 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:48.604042 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ebff3-848e-4da9-abb7-93ffbfc6782f None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:48.604333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ebff3-848e-4da9-abb7-93ffbfc6782f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:48.604587 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ebff3-848e-4da9-abb7-93ffbfc6782f None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:48.606168 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 447/883] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:18:48.610873 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48efbcb-5eed-4590-823f-3552380a7507 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:48.611152 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48efbcb-5eed-4590-823f-3552380a7507 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:48.611425 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48efbcb-5eed-4590-823f-3552380a7507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:48.630394 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a48efbcb-5eed-4590-823f-3552380a7507 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:48.631598 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a48efbcb-5eed-4590-823f-3552380a7507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:48.676462 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:48.693122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:48.703227 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93917d4d-08fd-4c7c-b042-85da6f166b23 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:48.710105 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 448/884] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:18:48.739598 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d34b8-18df-4b0d-b802-d2607c5dd19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:48.740449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d34b8-18df-4b0d-b802-d2607c5dd19c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:48.741527 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d34b8-18df-4b0d-b802-d2607c5dd19c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:48.741991 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d34b8-18df-4b0d-b802-d2607c5dd19c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:48.750180 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d34b8-18df-4b0d-b802-d2607c5dd19c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:48.751802 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d34b8-18df-4b0d-b802-d2607c5dd19c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:48.763989 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-2d4d34b8-18df-4b0d-b802-d2607c5dd19c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:18:48.763989 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:48.763989 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:48.763989 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:48.763989 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:48.763989 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:48.763989 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:48.763989 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:48.763989 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:48.763989 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:48.763989 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:48.763989 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:48.763989 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:48.763989 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:48.763989 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:48.763989 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:48.765178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:48.765178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:48.765178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:48.765178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:48.765178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:48.765178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:48.765178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:48.765178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:48.765178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:48.765178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:48.765178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:48.765178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:48.765178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:48.765178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:48.765178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:48.765178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:48.765178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:48.765178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:48.766499 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:48.766499 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:48.766499 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:48.766499 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:48.766499 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:48.766499 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:48.766499 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:48.766499 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:48.766499 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:18:48.766499 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:48.768685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 437/885] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Sep 1 16:18:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:48.775129 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48efbcb-5eed-4590-823f-3552380a7507 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:48.880179 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48efbcb-5eed-4590-823f-3552380a7507 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:48.920340 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5604957-7aa1-49e0-830e-e0e3cdf10bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:48.920999 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5604957-7aa1-49e0-830e-e0e3cdf10bea None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:48.921279 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5604957-7aa1-49e0-830e-e0e3cdf10bea None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:48.921549 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5604957-7aa1-49e0-830e-e0e3cdf10bea None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:48.922324 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5604957-7aa1-49e0-830e-e0e3cdf10bea None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:48.922631 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5604957-7aa1-49e0-830e-e0e3cdf10bea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:48.924056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5604957-7aa1-49e0-830e-e0e3cdf10bea None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:48.943074 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 449/886] 10.222.0.58 () {56 vars in 1149 bytes} [Thu Sep 1 16:18:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:48.949297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d34b8-18df-4b0d-b802-d2607c5dd19c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:49.016511 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5604957-7aa1-49e0-830e-e0e3cdf10bea None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:49.058096 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d34b8-18df-4b0d-b802-d2607c5dd19c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:49.077990 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:49.094405 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc937037-6530-49a0-a08a-f835e0872a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:49.095014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc937037-6530-49a0-a08a-f835e0872a98 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:49.095296 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc937037-6530-49a0-a08a-f835e0872a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:49.095562 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc937037-6530-49a0-a08a-f835e0872a98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:49.105218 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:49.135113 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5604957-7aa1-49e0-830e-e0e3cdf10bea None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:49.167337 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc937037-6530-49a0-a08a-f835e0872a98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:49.168797 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc937037-6530-49a0-a08a-f835e0872a98 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:49.173688 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a7ca8-4e21-4e16-81ee-afcba78397a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:49.174348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a7ca8-4e21-4e16-81ee-afcba78397a5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:49.174627 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a7ca8-4e21-4e16-81ee-afcba78397a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:49.174895 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a7ca8-4e21-4e16-81ee-afcba78397a5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:49.175471 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543a7ca8-4e21-4e16-81ee-afcba78397a5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:49.177239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543a7ca8-4e21-4e16-81ee-afcba78397a5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:49.187408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-543a7ca8-4e21-4e16-81ee-afcba78397a5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:18:49.187408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:49.187408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:18:49.187408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:18:49.187408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:18:49.187408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:18:49.187408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:18:49.187408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:49.187408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:18:49.187408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:49.187408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:18:49.187408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:49.187408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:49.187408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:49.187408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:49.187408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:49.187408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:49.188414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:49.188414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:49.188414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:49.188414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:49.188414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:49.188414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:49.188414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:18:49.188414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:18:49.188414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:18:49.188414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:18:49.188414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:49.188414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:49.188414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:49.188414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:49.188414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:49.188414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:49.189278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:49.189278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:49.189278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:49.189278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:49.189278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:49.189278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:49.189278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:49.189278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:49.189278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:49.189278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:49.189278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:18:49.189278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:49.189278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:18:49.189278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:49.193444 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 450/887] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Sep 1 16:18:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:49.194746 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa395672-9530-4151-9c6c-d019a8fa0b3c None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:49.195029 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa395672-9530-4151-9c6c-d019a8fa0b3c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:49.195299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa395672-9530-4151-9c6c-d019a8fa0b3c None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:49.200634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 451/888] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:49.201381 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa395672-9530-4151-9c6c-d019a8fa0b3c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:49.270643 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 438/889] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:18:48 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:18:49.271910 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e5b71-2463-4322-bfc2-35e18509a294 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:49.272219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e5b71-2463-4322-bfc2-35e18509a294 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:49.272494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e5b71-2463-4322-bfc2-35e18509a294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:49.291177 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fe8e5b71-2463-4322-bfc2-35e18509a294 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:49.292626 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fe8e5b71-2463-4322-bfc2-35e18509a294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:49.311149 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa395672-9530-4151-9c6c-d019a8fa0b3c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:49.325933 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:49.326210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:49.347315 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a10326-b57a-4e1f-8461-9338024a13bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:49.347908 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a10326-b57a-4e1f-8461-9338024a13bc None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:49.348181 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a10326-b57a-4e1f-8461-9338024a13bc None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:49.348446 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a10326-b57a-4e1f-8461-9338024a13bc None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:49.349076 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a10326-b57a-4e1f-8461-9338024a13bc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:49.349354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a10326-b57a-4e1f-8461-9338024a13bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:49.351237 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a10326-b57a-4e1f-8461-9338024a13bc None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:49.355635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 452/890] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Sep 1 16:18:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:49.356736 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d371df37-9370-4f4c-8722-aee983733f75 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:49.357015 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d371df37-9370-4f4c-8722-aee983733f75 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:49.357286 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d371df37-9370-4f4c-8722-aee983733f75 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:49.358478 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 453/891] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:49.359157 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d371df37-9370-4f4c-8722-aee983733f75 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:49.369035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 439/892] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:49 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:18:49.369872 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e5b71-2463-4322-bfc2-35e18509a294 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:49.471225 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d371df37-9370-4f4c-8722-aee983733f75 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:49.480226 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e5b71-2463-4322-bfc2-35e18509a294 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:49.507590 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c198a-10eb-475d-aaab-584d55b2e9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:49.508309 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c198a-10eb-475d-aaab-584d55b2e9c2 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:49.508586 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c198a-10eb-475d-aaab-584d55b2e9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:49.508860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c198a-10eb-475d-aaab-584d55b2e9c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:49.515035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d2dce-a3a1-4ee3-8eff-52156279814a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:49.515640 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d2dce-a3a1-4ee3-8eff-52156279814a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:49.516154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d2dce-a3a1-4ee3-8eff-52156279814a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:49.516269 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d2dce-a3a1-4ee3-8eff-52156279814a None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:49.516810 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d2dce-a3a1-4ee3-8eff-52156279814a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:49.517115 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d2dce-a3a1-4ee3-8eff-52156279814a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:49.518500 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d2dce-a3a1-4ee3-8eff-52156279814a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:49.519645 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8c198a-10eb-475d-aaab-584d55b2e9c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:49.521071 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8c198a-10eb-475d-aaab-584d55b2e9c2 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:49.524145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 440/893] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Sep 1 16:18:49 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:49.529858 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a144853-f0cf-41cc-913f-1311967951bb None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:49.530151 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a144853-f0cf-41cc-913f-1311967951bb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:49.530713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a144853-f0cf-41cc-913f-1311967951bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:49.536715 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-6e8c198a-10eb-475d-aaab-584d55b2e9c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:49.536715 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:49.536715 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:49.536715 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:49.536715 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:49.536715 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:49.536715 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:49.536715 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:49.536715 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:49.536715 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:49.536715 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:49.536715 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:49.536715 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:49.536715 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:49.536715 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:49.536715 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:49.538583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:49.538583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:49.538583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:49.538583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:49.538583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:49.538583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:49.538583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:49.538583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:49.538583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:49.538583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:49.538583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:49.538583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:49.538583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:49.538583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:49.538583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:49.538583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:49.538583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:49.539499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:49.539499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:49.539499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:49.539499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 454/894] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:49.539499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c198a-10eb-475d-aaab-584d55b2e9c2 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:49.553667 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1a144853-f0cf-41cc-913f-1311967951bb None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:49.554894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1a144853-f0cf-41cc-913f-1311967951bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:49.570506 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:49.583255 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:49.626013 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 441/895] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:49 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:18:49.627228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a144853-f0cf-41cc-913f-1311967951bb None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:49.653298 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c198a-10eb-475d-aaab-584d55b2e9c2 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:49.689931 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5bf83-3d9b-4f67-bf5d-28daa1c7524b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:49.690612 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5bf83-3d9b-4f67-bf5d-28daa1c7524b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:49.690883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5bf83-3d9b-4f67-bf5d-28daa1c7524b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:49.691105 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5bf83-3d9b-4f67-bf5d-28daa1c7524b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:49.691804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5bf83-3d9b-4f67-bf5d-28daa1c7524b None admin] RBAC: Adding query filter params (interface=public, service_id=52b8d7d67d3d40409e732bd6a0f3af06, region_id=RegionOne) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:49.692122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5bf83-3d9b-4f67-bf5d-28daa1c7524b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:49.693469 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5bf83-3d9b-4f67-bf5d-28daa1c7524b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:49.698649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 455/896] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Sep 1 16:18:49 2022] GET /identity/v3/endpoints?service_id=52b8d7d67d3d40409e732bd6a0f3af06&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:49.703818 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5bf83-3d9b-4f67-bf5d-28daa1c7524b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:49.731990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a144853-f0cf-41cc-913f-1311967951bb None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:49.768810 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cde7b6-246f-45a5-9fc2-a680e89479b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:49.769422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cde7b6-246f-45a5-9fc2-a680e89479b5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:49.769701 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cde7b6-246f-45a5-9fc2-a680e89479b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:49.769970 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cde7b6-246f-45a5-9fc2-a680e89479b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:49.770638 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cde7b6-246f-45a5-9fc2-a680e89479b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:49.770940 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cde7b6-246f-45a5-9fc2-a680e89479b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:49.772339 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cde7b6-246f-45a5-9fc2-a680e89479b5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:49.783409 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 442/897] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:49.784196 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cde7b6-246f-45a5-9fc2-a680e89479b5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:49.807516 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:49.811021 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:49.820154 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5bf83-3d9b-4f67-bf5d-28daa1c7524b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:49.857292 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab54c2-8cc6-44f1-9d44-735b47c41594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:49.857885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab54c2-8cc6-44f1-9d44-735b47c41594 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:49.858155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab54c2-8cc6-44f1-9d44-735b47c41594 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:49.858427 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab54c2-8cc6-44f1-9d44-735b47c41594 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:49.859016 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab54c2-8cc6-44f1-9d44-735b47c41594 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:49.860874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab54c2-8cc6-44f1-9d44-735b47c41594 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:49.869998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-c2ab54c2-8cc6-44f1-9d44-735b47c41594 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:18:49.869998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:49.869998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:18:49.869998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:18:49.869998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:18:49.869998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:18:49.869998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:18:49.869998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:49.869998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:18:49.869998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:49.869998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:18:49.869998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:49.869998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:49.869998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:49.869998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:49.869998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:49.869998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:49.871028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:49.871028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:49.871028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:49.871028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:49.871028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:49.871028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:49.871028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:18:49.871028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:18:49.871028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:18:49.871028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:18:49.871028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:49.871028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:49.871028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:49.871028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:49.871028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:49.871028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:49.871885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:49.871885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:49.871885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:49.871885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:49.871885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:49.871885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:49.871885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:49.871885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:49.871885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:49.871885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:49.871885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:18:49.871885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:49.871885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:18:49.871885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:49.871885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 456/898] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Sep 1 16:18:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:49.873357 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab54c2-8cc6-44f1-9d44-735b47c41594 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:49.896944 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cde7b6-246f-45a5-9fc2-a680e89479b5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:49.932364 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c481a4-7c05-4ed4-bf63-66e421f6f8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:49.932965 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c481a4-7c05-4ed4-bf63-66e421f6f8b7 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:49.933248 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c481a4-7c05-4ed4-bf63-66e421f6f8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:49.933528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c481a4-7c05-4ed4-bf63-66e421f6f8b7 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:49.934128 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c481a4-7c05-4ed4-bf63-66e421f6f8b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:49.935521 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c481a4-7c05-4ed4-bf63-66e421f6f8b7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:49.941382 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 443/899] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Sep 1 16:18:49 2022] GET /identity/v3/services => generated 1268 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:18:49.942082 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c481a4-7c05-4ed4-bf63-66e421f6f8b7 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:49.977297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab54c2-8cc6-44f1-9d44-735b47c41594 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:50.014595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a26b9dc-d712-4caa-bbb5-cd9691483c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:50.015265 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26b9dc-d712-4caa-bbb5-cd9691483c78 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:50.015554 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26b9dc-d712-4caa-bbb5-cd9691483c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:50.015934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26b9dc-d712-4caa-bbb5-cd9691483c78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:50.023510 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a26b9dc-d712-4caa-bbb5-cd9691483c78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:50.024976 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a26b9dc-d712-4caa-bbb5-cd9691483c78 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:50.035856 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:50.037012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-8a26b9dc-d712-4caa-bbb5-cd9691483c78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:50.037012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:50.037012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:50.037012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:50.037012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:50.037012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:50.037012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:50.037012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:50.037012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:50.037012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:50.037012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:50.037012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:50.037012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:50.037012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:50.037012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:50.038473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:50.038473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:50.038473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:50.038473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:50.038473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:50.038473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:50.038473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:50.038473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:50.038473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:50.038473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:50.038473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:50.038473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:50.038473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:50.038473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:50.038473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:50.038473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:50.038473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:50.040214 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:50.040214 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:50.040214 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:50.040214 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:50.040214 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:50.040214 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:50.040214 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:50.040214 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:50.040214 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 457/900] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:50.040214 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a26b9dc-d712-4caa-bbb5-cd9691483c78 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:50.063225 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:50.070167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c481a4-7c05-4ed4-bf63-66e421f6f8b7 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:50.111430 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18849146-67bb-408b-827e-5aca1212f47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:50.111825 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18849146-67bb-408b-827e-5aca1212f47e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:50.112106 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18849146-67bb-408b-827e-5aca1212f47e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:50.112387 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18849146-67bb-408b-827e-5aca1212f47e None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:50.113030 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18849146-67bb-408b-827e-5aca1212f47e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:50.113422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18849146-67bb-408b-827e-5aca1212f47e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:50.114885 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18849146-67bb-408b-827e-5aca1212f47e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:50.120080 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 444/901] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Sep 1 16:18:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:50.120861 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18849146-67bb-408b-827e-5aca1212f47e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:50.167494 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a26b9dc-d712-4caa-bbb5-cd9691483c78 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:50.203612 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53477928-e3f3-4c43-b088-1316a0236881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:50.204235 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53477928-e3f3-4c43-b088-1316a0236881 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:50.204515 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53477928-e3f3-4c43-b088-1316a0236881 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:50.204783 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53477928-e3f3-4c43-b088-1316a0236881 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:50.205366 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53477928-e3f3-4c43-b088-1316a0236881 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:50.206758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53477928-e3f3-4c43-b088-1316a0236881 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:50.215932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-53477928-e3f3-4c43-b088-1316a0236881 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:18:50.215932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:50.215932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:18:50.215932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:18:50.215932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:18:50.215932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:18:50.215932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:18:50.215932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:50.215932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:18:50.215932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:50.215932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:18:50.215932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:50.215932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:50.215932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:50.215932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:50.215932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:50.216903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:50.216903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:50.216903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:50.216903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:50.216903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:50.216903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:50.216903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:50.216903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:18:50.216903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:18:50.216903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:18:50.216903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:18:50.216903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:50.216903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:50.216903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:50.216903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:50.216903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:50.216903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:50.217818 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:50.217818 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:50.217818 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:50.217818 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:50.217818 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:50.217818 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:50.217818 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:50.217818 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:50.217818 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:50.217818 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:50.217818 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:18:50.217818 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:50.217818 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:18:50.217818 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:50.217818 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 458/902] 10.222.0.58 () {56 vars in 1132 bytes} [Thu Sep 1 16:18:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:50.218720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53477928-e3f3-4c43-b088-1316a0236881 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:50.232312 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18849146-67bb-408b-827e-5aca1212f47e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:50.268068 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0d54a0-590a-4781-9973-e246bd3d72be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:50.268869 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d54a0-590a-4781-9973-e246bd3d72be None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:50.269147 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d54a0-590a-4781-9973-e246bd3d72be None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:50.269416 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d54a0-590a-4781-9973-e246bd3d72be None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:50.270242 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0d54a0-590a-4781-9973-e246bd3d72be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:50.270551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0d54a0-590a-4781-9973-e246bd3d72be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:50.271977 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0d54a0-590a-4781-9973-e246bd3d72be None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:50.284413 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 445/903] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:50.285214 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0d54a0-590a-4781-9973-e246bd3d72be None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:50.299779 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:50.319456 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:50.342239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53477928-e3f3-4c43-b088-1316a0236881 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:50.378167 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847aeb1-efe8-4953-bfbd-f6fe6c93b4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:50.378713 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847aeb1-efe8-4953-bfbd-f6fe6c93b4d0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:50.378988 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847aeb1-efe8-4953-bfbd-f6fe6c93b4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:50.379254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847aeb1-efe8-4953-bfbd-f6fe6c93b4d0 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:50.380325 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847aeb1-efe8-4953-bfbd-f6fe6c93b4d0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:50.380636 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847aeb1-efe8-4953-bfbd-f6fe6c93b4d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:50.382639 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847aeb1-efe8-4953-bfbd-f6fe6c93b4d0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:50.387076 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 459/904] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Sep 1 16:18:50 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:50.387845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847aeb1-efe8-4953-bfbd-f6fe6c93b4d0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:50.403340 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0d54a0-590a-4781-9973-e246bd3d72be None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:50.443745 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e6182-769e-45a6-af4b-cd4f2e5591af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:50.445332 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e6182-769e-45a6-af4b-cd4f2e5591af None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:50.445714 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e6182-769e-45a6-af4b-cd4f2e5591af None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:50.446089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e6182-769e-45a6-af4b-cd4f2e5591af None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:50.446897 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e6182-769e-45a6-af4b-cd4f2e5591af None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:50.447303 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e6182-769e-45a6-af4b-cd4f2e5591af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:50.448808 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e6182-769e-45a6-af4b-cd4f2e5591af None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:50.454767 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 446/905] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Sep 1 16:18:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:50.460060 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e6182-769e-45a6-af4b-cd4f2e5591af None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:50.513778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847aeb1-efe8-4953-bfbd-f6fe6c93b4d0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:50.559542 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe36b93-c178-4d14-b180-5aaacaa66b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:50.560305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe36b93-c178-4d14-b180-5aaacaa66b68 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:50.560532 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe36b93-c178-4d14-b180-5aaacaa66b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:50.560799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe36b93-c178-4d14-b180-5aaacaa66b68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:50.561576 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe36b93-c178-4d14-b180-5aaacaa66b68 None admin] RBAC: Adding query filter params (role.id=1ff98ccdce644654b43e48dc28fce36b, scope.project.id=479530ced580426b85d1fc07af43417f, user.id=dd7b98536fa445cebd552fcbf54b757d) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:50.561885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe36b93-c178-4d14-b180-5aaacaa66b68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:50.563495 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe36b93-c178-4d14-b180-5aaacaa66b68 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:50.570247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 460/906] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:18:50 2022] GET /identity/v3/role_assignments?user.id=dd7b98536fa445cebd552fcbf54b757d&scope.project.id=479530ced580426b85d1fc07af43417f&role.id=1ff98ccdce644654b43e48dc28fce36b => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:50.572864 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:50.605855 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e6182-769e-45a6-af4b-cd4f2e5591af None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:50.646897 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155c099-a485-4fd8-87ac-481a75ce94ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:50.647479 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155c099-a485-4fd8-87ac-481a75ce94ae None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:50.647759 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155c099-a485-4fd8-87ac-481a75ce94ae None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:50.648034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155c099-a485-4fd8-87ac-481a75ce94ae None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:50.648684 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155c099-a485-4fd8-87ac-481a75ce94ae None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:50.648993 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155c099-a485-4fd8-87ac-481a75ce94ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:50.650415 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155c099-a485-4fd8-87ac-481a75ce94ae None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:50.656634 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 447/907] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Sep 1 16:18:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:18:50.662161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe36b93-c178-4d14-b180-5aaacaa66b68 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:50.675916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:50.777971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe36b93-c178-4d14-b180-5aaacaa66b68 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:50.812793 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8652c12-f2da-4863-9a4c-46a1a05d19e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:50.813632 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8652c12-f2da-4863-9a4c-46a1a05d19e2 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:50.814055 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8652c12-f2da-4863-9a4c-46a1a05d19e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:50.814491 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8652c12-f2da-4863-9a4c-46a1a05d19e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:50.815342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8652c12-f2da-4863-9a4c-46a1a05d19e2 None admin] RBAC: Adding query filter params (interface=public, service_id=197ca0b1fdec4c1bae6dbe51167c992d, region_id=RegionOne) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:50.815812 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8652c12-f2da-4863-9a4c-46a1a05d19e2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:50.817306 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8652c12-f2da-4863-9a4c-46a1a05d19e2 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:50.823794 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 461/908] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Sep 1 16:18:50 2022] GET /identity/v3/endpoints?service_id=197ca0b1fdec4c1bae6dbe51167c992d&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:50.831922 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155c099-a485-4fd8-87ac-481a75ce94ae None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:50.935484 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155c099-a485-4fd8-87ac-481a75ce94ae None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:50.970656 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a28fcf0-c064-448f-bca8-ff1f76e623f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:50.971264 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28fcf0-c064-448f-bca8-ff1f76e623f8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:50.971548 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28fcf0-c064-448f-bca8-ff1f76e623f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:50.971824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28fcf0-c064-448f-bca8-ff1f76e623f8 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:50.972438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a28fcf0-c064-448f-bca8-ff1f76e623f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:50.973803 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a28fcf0-c064-448f-bca8-ff1f76e623f8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:50.980344 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 448/909] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Sep 1 16:18:50 2022] GET /identity/v3/services => generated 1268 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:18:51.956420 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcb7d0-816b-4cfa-bfa7-5f8ba1ca1cdb None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:51.956975 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcb7d0-816b-4cfa-bfa7-5f8ba1ca1cdb None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:51.956975 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcb7d0-816b-4cfa-bfa7-5f8ba1ca1cdb None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:51.958642 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 462/910] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:51.966232 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263bd13-e2d0-4e46-bee2-48ee19fe6104 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:51.966232 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263bd13-e2d0-4e46-bee2-48ee19fe6104 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:51.967248 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263bd13-e2d0-4e46-bee2-48ee19fe6104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:51.976557 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:51.993386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4263bd13-e2d0-4e46-bee2-48ee19fe6104 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:51.994622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4263bd13-e2d0-4e46-bee2-48ee19fe6104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:52.085916 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 449/911] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:51 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:18:52.377857 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcb7d0-816b-4cfa-bfa7-5f8ba1ca1cdb None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:52.393663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0404a-c176-4b1c-a261-768afba9aaad None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:52.394359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0404a-c176-4b1c-a261-768afba9aaad None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:52.394868 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0404a-c176-4b1c-a261-768afba9aaad None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:52.396554 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 450/912] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:18:52.401688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f58528-12f2-4a81-95c7-931391b42d3e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:52.402251 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f58528-12f2-4a81-95c7-931391b42d3e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:52.402743 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f58528-12f2-4a81-95c7-931391b42d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:52.422438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f3f58528-12f2-4a81-95c7-931391b42d3e None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:52.423969 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f3f58528-12f2-4a81-95c7-931391b42d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:52.460381 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:52.499016 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcb7d0-816b-4cfa-bfa7-5f8ba1ca1cdb None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:52.506349 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 451/913] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:52 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:18:52.534217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba9aff-b3a3-41f7-9ab6-f584dd74fadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:52.534858 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba9aff-b3a3-41f7-9ab6-f584dd74fadb None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:52.535137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba9aff-b3a3-41f7-9ab6-f584dd74fadb None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:52.535405 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba9aff-b3a3-41f7-9ab6-f584dd74fadb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:52.535991 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ba9aff-b3a3-41f7-9ab6-f584dd74fadb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:52.537372 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ba9aff-b3a3-41f7-9ab6-f584dd74fadb None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:52.546701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-34ba9aff-b3a3-41f7-9ab6-f584dd74fadb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:18:52.546701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:52.546701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:18:52.546701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:18:52.546701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:18:52.546701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:18:52.546701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:18:52.546701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:52.546701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:18:52.546701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:52.546701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:18:52.546701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:52.546701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:52.546701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:52.546701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:52.546701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:52.546701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:52.547826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:52.547826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:52.547826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:52.547826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:52.547826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:52.547826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:52.547826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:18:52.547826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:18:52.547826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:18:52.547826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:18:52.547826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:52.547826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:52.547826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:52.547826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:52.547826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:52.547826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:52.548751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:52.548751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:52.548751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:52.548751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:52.548751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:52.548751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:52.548751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:52.548751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:52.548751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:52.548751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:52.548751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:18:52.548751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:52.548751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:18:52.548751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:52.548751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 463/914] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Sep 1 16:18:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:52.554427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f58528-12f2-4a81-95c7-931391b42d3e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:52.570585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b52460-4718-4905-b9ac-d1e4872422f3 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:52.570859 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b52460-4718-4905-b9ac-d1e4872422f3 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:52.571130 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b52460-4718-4905-b9ac-d1e4872422f3 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:52.572814 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 464/915] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:18:52.577960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a64833-d31e-475d-a5de-95df93d54b5d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:52.578273 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a64833-d31e-475d-a5de-95df93d54b5d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:52.578540 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a64833-d31e-475d-a5de-95df93d54b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:52.597890 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-74a64833-d31e-475d-a5de-95df93d54b5d None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:52.599119 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-74a64833-d31e-475d-a5de-95df93d54b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:52.623332 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:52.664486 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f58528-12f2-4a81-95c7-931391b42d3e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:52.678765 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 465/916] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:52 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:18:52.705551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b314238-5d9b-47f4-8e1f-d37edb28d946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:52.706542 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b314238-5d9b-47f4-8e1f-d37edb28d946 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:52.707013 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b314238-5d9b-47f4-8e1f-d37edb28d946 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:52.707476 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b314238-5d9b-47f4-8e1f-d37edb28d946 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:52.708293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b314238-5d9b-47f4-8e1f-d37edb28d946 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:52.708774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b314238-5d9b-47f4-8e1f-d37edb28d946 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:52.710377 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b314238-5d9b-47f4-8e1f-d37edb28d946 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:52.715486 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 452/917] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Sep 1 16:18:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:52.721224 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a64833-d31e-475d-a5de-95df93d54b5d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:52.783960 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b314238-5d9b-47f4-8e1f-d37edb28d946 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:52.793668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:52.827511 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a64833-d31e-475d-a5de-95df93d54b5d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:52.863706 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e392b-2bcb-4cd5-a5cd-1e636c703775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:52.864312 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e392b-2bcb-4cd5-a5cd-1e636c703775 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:52.864595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e392b-2bcb-4cd5-a5cd-1e636c703775 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:52.864866 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e392b-2bcb-4cd5-a5cd-1e636c703775 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:52.865499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e392b-2bcb-4cd5-a5cd-1e636c703775 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:52.865880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e392b-2bcb-4cd5-a5cd-1e636c703775 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:52.867811 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e392b-2bcb-4cd5-a5cd-1e636c703775 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:52.877762 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 466/918] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Sep 1 16:18:52 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:18:52.878623 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e392b-2bcb-4cd5-a5cd-1e636c703775 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:52.918637 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b314238-5d9b-47f4-8e1f-d37edb28d946 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:52.953602 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:52.965180 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a1686-cdc8-4417-b7cf-6ace36bc7484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:52.966062 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a1686-cdc8-4417-b7cf-6ace36bc7484 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:52.966377 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a1686-cdc8-4417-b7cf-6ace36bc7484 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:52.966918 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a1686-cdc8-4417-b7cf-6ace36bc7484 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:52.967563 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a1686-cdc8-4417-b7cf-6ace36bc7484 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:52.969700 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a1686-cdc8-4417-b7cf-6ace36bc7484 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:52.987157 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e392b-2bcb-4cd5-a5cd-1e636c703775 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:52.998580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 453/919] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Sep 1 16:18:52 2022] POST /identity/v3/services => generated 241 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:18:53.000510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a1686-cdc8-4417-b7cf-6ace36bc7484 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:53.023694 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:53.038138 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81c187-a648-4638-8fdb-3f2815eb5476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:53.038801 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81c187-a648-4638-8fdb-3f2815eb5476 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:53.039110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81c187-a648-4638-8fdb-3f2815eb5476 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:53.039356 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81c187-a648-4638-8fdb-3f2815eb5476 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:53.040055 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea81c187-a648-4638-8fdb-3f2815eb5476 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:53.041466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea81c187-a648-4638-8fdb-3f2815eb5476 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:53.065776 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 467/920] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:18:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:18:53.067173 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a76079-62b7-4c2b-85cb-8039cea998b1 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:53.067467 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a76079-62b7-4c2b-85cb-8039cea998b1 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:53.067746 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a76079-62b7-4c2b-85cb-8039cea998b1 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:53.069409 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 468/921] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:53.075167 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc5cde-5e4e-475b-8803-8b251475bd8f None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:53.075529 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc5cde-5e4e-475b-8803-8b251475bd8f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:53.075757 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc5cde-5e4e-475b-8803-8b251475bd8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:53.104332 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a7bc5cde-5e4e-475b-8803-8b251475bd8f None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:53.106761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a7bc5cde-5e4e-475b-8803-8b251475bd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:53.149909 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a1686-cdc8-4417-b7cf-6ace36bc7484 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:53.201758 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9170c5f-e3f0-4077-bb31-5c9f64e5297a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:53.202390 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9170c5f-e3f0-4077-bb31-5c9f64e5297a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:53.202677 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 469/922] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:18:53.202800 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9170c5f-e3f0-4077-bb31-5c9f64e5297a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:53.203024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9170c5f-e3f0-4077-bb31-5c9f64e5297a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:53.218188 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9170c5f-e3f0-4077-bb31-5c9f64e5297a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:53.223002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-f9170c5f-e3f0-4077-bb31-5c9f64e5297a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:18:53.223002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:53.223002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:53.223002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:53.223002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:53.223002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:53.223002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:53.223002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:53.223002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:53.223002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:53.223002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:53.223002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:53.223002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:53.223002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:53.223002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:53.223002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:53.224054 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:53.224054 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:53.224054 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:53.224054 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:53.224054 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:53.224054 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:53.224054 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:53.224054 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:53.224054 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:53.224054 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:53.224054 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:53.224054 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:53.224054 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:53.224054 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:53.224054 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:53.224054 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:53.225041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:53.225041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:53.225041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:53.225041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:53.225041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:53.225041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:53.225041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:53.225041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:53.225041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:53.225041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:53.225041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:18:53.225041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:53.225041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 454/923] 10.222.0.58 () {56 vars in 1120 bytes} [Thu Sep 1 16:18:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:53.230961 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc5cde-5e4e-475b-8803-8b251475bd8f None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:53.335558 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc5cde-5e4e-475b-8803-8b251475bd8f None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:53.345596 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:53.374017 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639ef9e-9dcf-4ebb-9ce2-7f2090109bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:53.374656 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639ef9e-9dcf-4ebb-9ce2-7f2090109bef None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:53.374938 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639ef9e-9dcf-4ebb-9ce2-7f2090109bef None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:53.375213 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639ef9e-9dcf-4ebb-9ce2-7f2090109bef None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:53.375946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a639ef9e-9dcf-4ebb-9ce2-7f2090109bef None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:53.376254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a639ef9e-9dcf-4ebb-9ce2-7f2090109bef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:53.377665 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a639ef9e-9dcf-4ebb-9ce2-7f2090109bef None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:53.387367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 470/924] 10.222.0.58 () {56 vars in 1115 bytes} [Thu Sep 1 16:18:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:53.393612 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9170c5f-e3f0-4077-bb31-5c9f64e5297a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:53.443875 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:53.482545 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639ef9e-9dcf-4ebb-9ce2-7f2090109bef None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:53.505943 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9170c5f-e3f0-4077-bb31-5c9f64e5297a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:53.545432 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6319f-b4d2-48c9-8b92-3e95b4feb7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:53.546059 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6319f-b4d2-48c9-8b92-3e95b4feb7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:53.546390 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6319f-b4d2-48c9-8b92-3e95b4feb7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:53.546665 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6319f-b4d2-48c9-8b92-3e95b4feb7a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:53.557619 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6319f-b4d2-48c9-8b92-3e95b4feb7a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:53.559184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6319f-b4d2-48c9-8b92-3e95b4feb7a8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:53.574651 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-6af6319f-b4d2-48c9-8b92-3e95b4feb7a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:53.574651 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:53.574651 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:53.574651 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:53.574651 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:53.574651 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:53.574651 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:53.574651 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:53.574651 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:53.574651 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:53.574651 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:53.574651 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:53.574651 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:53.574651 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:53.574651 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:53.574651 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:53.575698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:53.575698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:53.575698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:53.575698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:53.575698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:53.575698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:53.575698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:53.575698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:53.575698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:53.575698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:53.575698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:53.575698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:53.575698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:53.575698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:53.575698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:53.575698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:53.575698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:53.576622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:53.576622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:53.576622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:53.576622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 455/925] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:53.583146 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6319f-b4d2-48c9-8b92-3e95b4feb7a8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:53.633839 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639ef9e-9dcf-4ebb-9ce2-7f2090109bef None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:53.679992 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9691b6-c64e-479f-b117-adae15e582a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:53.680870 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9691b6-c64e-479f-b117-adae15e582a0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:53.681147 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9691b6-c64e-479f-b117-adae15e582a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:53.681422 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9691b6-c64e-479f-b117-adae15e582a0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:53.682008 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9691b6-c64e-479f-b117-adae15e582a0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:53.684210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9691b6-c64e-479f-b117-adae15e582a0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:53.695239 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:53.696591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-6c9691b6-c64e-479f-b117-adae15e582a0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:18:53.696591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:53.696591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:18:53.696591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:18:53.696591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:18:53.696591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:18:53.696591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:18:53.696591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:53.696591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:18:53.696591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:53.696591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:18:53.696591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:53.696591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:53.696591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:53.696591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:53.696591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:53.697539 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:53.697539 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:53.697539 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:53.697539 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:53.697539 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:53.697539 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:53.697539 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:53.697539 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:18:53.697539 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:18:53.697539 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:18:53.697539 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:18:53.697539 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:53.697539 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:53.697539 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:53.697539 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:53.697539 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:53.697539 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:53.702950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:53.702950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:53.702950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:53.702950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:53.702950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:53.702950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:53.702950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:53.702950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:53.702950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:53.702950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:53.702950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:18:53.702950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:53.702950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:18:53.702950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:53.702950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 471/926] 10.222.0.58 () {56 vars in 1132 bytes} [Thu Sep 1 16:18:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:18:53.705167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6319f-b4d2-48c9-8b92-3e95b4feb7a8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:53.707394 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9691b6-c64e-479f-b117-adae15e582a0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:53.744547 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:53.752217 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4744b-7d83-402f-aff3-3f11efc24863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:53.753328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4744b-7d83-402f-aff3-3f11efc24863 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:53.753610 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4744b-7d83-402f-aff3-3f11efc24863 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:53.753882 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4744b-7d83-402f-aff3-3f11efc24863 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:53.754553 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4744b-7d83-402f-aff3-3f11efc24863 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:53.754856 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4744b-7d83-402f-aff3-3f11efc24863 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:53.757211 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4744b-7d83-402f-aff3-3f11efc24863 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:53.778281 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 456/927] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:53.780297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4744b-7d83-402f-aff3-3f11efc24863 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:53.824159 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9691b6-c64e-479f-b117-adae15e582a0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:53.864062 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b32969-09e3-4334-991f-d498a2a086ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:53.865105 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b32969-09e3-4334-991f-d498a2a086ff None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:53.866314 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b32969-09e3-4334-991f-d498a2a086ff None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:53.866867 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b32969-09e3-4334-991f-d498a2a086ff None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:53.868179 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b32969-09e3-4334-991f-d498a2a086ff None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:53.868990 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b32969-09e3-4334-991f-d498a2a086ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:53.872273 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b32969-09e3-4334-991f-d498a2a086ff None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:53.882562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 472/928] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Sep 1 16:18:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:18:53.885173 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b32969-09e3-4334-991f-d498a2a086ff None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:53.914477 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4744b-7d83-402f-aff3-3f11efc24863 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:53.951358 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191a561-3202-4c7f-b16a-9e9d8e84e354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:53.951997 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191a561-3202-4c7f-b16a-9e9d8e84e354 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:53.952409 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191a561-3202-4c7f-b16a-9e9d8e84e354 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:53.952705 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191a561-3202-4c7f-b16a-9e9d8e84e354 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:53.962594 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7191a561-3202-4c7f-b16a-9e9d8e84e354 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:53.965008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7191a561-3202-4c7f-b16a-9e9d8e84e354 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:53.979811 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-7191a561-3202-4c7f-b16a-9e9d8e84e354 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:18:53.979811 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:53.979811 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:53.979811 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:53.979811 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:53.979811 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:53.979811 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:53.979811 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:53.979811 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:53.979811 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:53.979811 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:53.979811 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:53.979811 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:53.979811 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:53.979811 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:53.979811 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:53.981703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:53.981703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:53.981703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:53.981703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:53.981703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:53.981703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:53.981703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:53.981703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:53.981703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:53.981703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:53.981703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:53.981703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:53.981703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:53.981703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:53.981703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:53.981703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:53.981703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:53.981703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:53.984316 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:53.984316 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:53.984316 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:53.984316 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:53.984316 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:53.984316 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:53.984316 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:53.984316 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:53.984316 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:18:53.984316 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:53.984316 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 457/929] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Sep 1 16:18:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:53.989983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191a561-3202-4c7f-b16a-9e9d8e84e354 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:53.998601 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:54.002884 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b32969-09e3-4334-991f-d498a2a086ff None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:54.013813 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:54.052697 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7d5c2-77eb-4d34-a0ac-7421c559b120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:54.053891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7d5c2-77eb-4d34-a0ac-7421c559b120 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:54.054434 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7d5c2-77eb-4d34-a0ac-7421c559b120 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:54.054915 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7d5c2-77eb-4d34-a0ac-7421c559b120 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:54.055771 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca7d5c2-77eb-4d34-a0ac-7421c559b120 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:54.056280 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca7d5c2-77eb-4d34-a0ac-7421c559b120 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:54.057841 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca7d5c2-77eb-4d34-a0ac-7421c559b120 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:54.070413 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 473/930] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Sep 1 16:18:53 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:54.077499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7d5c2-77eb-4d34-a0ac-7421c559b120 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:54.120708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191a561-3202-4c7f-b16a-9e9d8e84e354 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:54.161544 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f53bd6-c1e4-4133-bc98-992688e62fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:54.162164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f53bd6-c1e4-4133-bc98-992688e62fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:54.162476 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f53bd6-c1e4-4133-bc98-992688e62fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:54.163355 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f53bd6-c1e4-4133-bc98-992688e62fc2 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:54.164105 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f53bd6-c1e4-4133-bc98-992688e62fc2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:54.164416 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f53bd6-c1e4-4133-bc98-992688e62fc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:54.165966 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f53bd6-c1e4-4133-bc98-992688e62fc2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:54.189661 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 458/931] 10.222.0.58 () {56 vars in 1149 bytes} [Thu Sep 1 16:18:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:54.198923 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7d5c2-77eb-4d34-a0ac-7421c559b120 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:54.199281 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f53bd6-c1e4-4133-bc98-992688e62fc2 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:54.240069 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8fcc98-74de-46b4-bb3c-edee8bde37a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:54.240758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fcc98-74de-46b4-bb3c-edee8bde37a2 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:54.241124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fcc98-74de-46b4-bb3c-edee8bde37a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:54.241395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fcc98-74de-46b4-bb3c-edee8bde37a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:54.242063 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8fcc98-74de-46b4-bb3c-edee8bde37a2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:54.243578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8fcc98-74de-46b4-bb3c-edee8bde37a2 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:54.276116 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 474/932] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:18:54 2022] POST /identity/v3/endpoints => generated 347 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:18:54.304531 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:54.345395 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f53bd6-c1e4-4133-bc98-992688e62fc2 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:54.396309 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c0b33-859a-4eea-81e3-04fafb200a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:54.397222 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c0b33-859a-4eea-81e3-04fafb200a7f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:54.397344 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c0b33-859a-4eea-81e3-04fafb200a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:54.397539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c0b33-859a-4eea-81e3-04fafb200a7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:54.410471 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c0b33-859a-4eea-81e3-04fafb200a7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:54.411934 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c0b33-859a-4eea-81e3-04fafb200a7f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:54.433328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-b11c0b33-859a-4eea-81e3-04fafb200a7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:54.433328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:54.433328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:54.433328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:54.433328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:54.433328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:54.433328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:54.433328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:54.433328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:54.433328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:54.433328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:54.433328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:54.433328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:54.433328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:54.433328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:54.433328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:54.434378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:54.434378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:54.434378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:54.434378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:54.434378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:54.434378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:54.434378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:54.434378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:54.434378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:54.434378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:54.434378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:54.434378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:54.434378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:54.434378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:54.434378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:54.434378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:54.434378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:54.435311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:54.435311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:54.435311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:54.438355 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 459/933] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:54.440744 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8fcc98-74de-46b4-bb3c-edee8bde37a2 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:54.464318 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:54.559210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8fcc98-74de-46b4-bb3c-edee8bde37a2 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:54.597308 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252cd33-c68c-479d-bf2e-fd2f4c5b14b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:54.597931 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252cd33-c68c-479d-bf2e-fd2f4c5b14b5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:54.598235 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252cd33-c68c-479d-bf2e-fd2f4c5b14b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:54.598517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252cd33-c68c-479d-bf2e-fd2f4c5b14b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:54.599172 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5252cd33-c68c-479d-bf2e-fd2f4c5b14b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:54.599475 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5252cd33-c68c-479d-bf2e-fd2f4c5b14b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:54.601413 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5252cd33-c68c-479d-bf2e-fd2f4c5b14b5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:54.614466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 475/934] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:18:54.620854 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c0b33-859a-4eea-81e3-04fafb200a7f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:54.737862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c0b33-859a-4eea-81e3-04fafb200a7f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:54.758636 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:54.774821 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a16d94-8851-4714-b2f7-9ae04fb8473a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:54.775450 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a16d94-8851-4714-b2f7-9ae04fb8473a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:54.775727 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a16d94-8851-4714-b2f7-9ae04fb8473a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:54.775997 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a16d94-8851-4714-b2f7-9ae04fb8473a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:54.782918 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a16d94-8851-4714-b2f7-9ae04fb8473a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:54.784355 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a16d94-8851-4714-b2f7-9ae04fb8473a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:54.796551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-a1a16d94-8851-4714-b2f7-9ae04fb8473a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:54.796551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:54.796551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:54.796551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:54.796551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:54.796551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:54.796551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:54.796551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:54.796551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:54.796551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:54.796551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:54.796551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:54.796551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:54.796551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:54.796551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:54.797509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:54.797509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:54.797509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:54.797509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:54.797509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:54.797509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:54.797509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:54.797509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:54.797509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:54.797509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:54.797509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:54.797509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:54.797509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:54.797509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:54.797509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:54.797509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:54.797509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:54.798473 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:54.798473 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:54.798473 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:54.798473 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:54.798473 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:54.798473 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:54.798473 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:54.798473 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:54.802410 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 460/935] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:54.803781 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252cd33-c68c-479d-bf2e-fd2f4c5b14b5 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:54.886282 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:54.909095 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252cd33-c68c-479d-bf2e-fd2f4c5b14b5 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:54.945632 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449c347-f16f-4dd7-beb3-53222e33c2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:54.946259 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449c347-f16f-4dd7-beb3-53222e33c2e3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:54.946542 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449c347-f16f-4dd7-beb3-53222e33c2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:54.946813 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449c347-f16f-4dd7-beb3-53222e33c2e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:54.947509 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449c347-f16f-4dd7-beb3-53222e33c2e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:54.947813 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449c347-f16f-4dd7-beb3-53222e33c2e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:54.949230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449c347-f16f-4dd7-beb3-53222e33c2e3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:54.961512 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 476/936] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:54.967231 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a16d94-8851-4714-b2f7-9ae04fb8473a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:55.083703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a16d94-8851-4714-b2f7-9ae04fb8473a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:55.120141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdac1c5-4e6b-413b-b414-dd4227cff7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:55.120795 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdac1c5-4e6b-413b-b414-dd4227cff7d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:55.121072 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdac1c5-4e6b-413b-b414-dd4227cff7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:55.121340 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdac1c5-4e6b-413b-b414-dd4227cff7d3 None admin] PATH_INFO: `/v3/projects/479530ced580426b85d1fc07af43417f/users/dd7b98536fa445cebd552fcbf54b757d/roles/1ff98ccdce644654b43e48dc28fce36b` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:55.123525 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdac1c5-4e6b-413b-b414-dd4227cff7d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=479530ced580426b85d1fc07af43417f, user_id=dd7b98536fa445cebd552fcbf54b757d, role_id=1ff98ccdce644654b43e48dc28fce36b)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:55.125129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdac1c5-4e6b-413b-b414-dd4227cff7d3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:55.125893 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247394cf-bc4a-406f-8118-9794efd470b8 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:55.126369 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247394cf-bc4a-406f-8118-9794efd470b8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:55.126543 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247394cf-bc4a-406f-8118-9794efd470b8 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:55.128082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 477/937] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:18:55.133196 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32552e58-cf5f-47a3-8eba-df5367550a26 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:55.133506 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32552e58-cf5f-47a3-8eba-df5367550a26 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:55.133800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32552e58-cf5f-47a3-8eba-df5367550a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:55.146391 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 461/938] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Sep 1 16:18:54 2022] PUT /identity/v3/projects/479530ced580426b85d1fc07af43417f/users/dd7b98536fa445cebd552fcbf54b757d/roles/1ff98ccdce644654b43e48dc28fce36b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:18:55.154551 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-32552e58-cf5f-47a3-8eba-df5367550a26 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:55.155847 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-32552e58-cf5f-47a3-8eba-df5367550a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:55.170755 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec1f91-7b0c-4948-926b-68e00088427e None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:55.171421 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec1f91-7b0c-4948-926b-68e00088427e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:55.171914 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec1f91-7b0c-4948-926b-68e00088427e None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:55.173670 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 462/939] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:18:55.180808 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ff441-64fd-491a-8c7c-39c862bf0933 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:55.181372 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ff441-64fd-491a-8c7c-39c862bf0933 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:55.181651 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ff441-64fd-491a-8c7c-39c862bf0933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:55.200936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b87ff441-64fd-491a-8c7c-39c862bf0933 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:55.202726 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b87ff441-64fd-491a-8c7c-39c862bf0933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:55.250263 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 478/940] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:18:55.253081 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:55.291347 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 463/941] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:18:55.535461 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32552e58-cf5f-47a3-8eba-df5367550a26 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:55.596785 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ff441-64fd-491a-8c7c-39c862bf0933 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:55.604965 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:55.641291 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32552e58-cf5f-47a3-8eba-df5367550a26 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:55.678869 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53457ea2-df5b-4208-8163-e26b3551a5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:55.679864 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53457ea2-df5b-4208-8163-e26b3551a5ec None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:55.680348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53457ea2-df5b-4208-8163-e26b3551a5ec None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:55.680828 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53457ea2-df5b-4208-8163-e26b3551a5ec None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:55.681626 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53457ea2-df5b-4208-8163-e26b3551a5ec None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:55.683214 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53457ea2-df5b-4208-8163-e26b3551a5ec None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:55.695701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-53457ea2-df5b-4208-8163-e26b3551a5ec None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:18:55.695701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:55.695701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:18:55.695701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:18:55.695701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:18:55.695701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:18:55.695701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:18:55.695701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:55.695701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:18:55.695701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:55.695701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:18:55.695701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:55.695701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:55.695701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:55.695701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:55.695701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:55.695701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:55.696759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:55.696759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:55.696759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:55.696759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:55.696759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:55.696759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:55.696759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:18:55.696759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:18:55.696759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:18:55.696759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:18:55.696759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:55.696759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:55.696759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:55.696759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:55.696759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:55.696759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:55.697622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:55.697622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:55.697622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:55.697622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:55.697622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:55.697622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:55.697622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:55.697622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:55.697622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:55.697622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:55.697622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:18:55.697622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:55.697622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:18:55.697622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:55.705170 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 479/942] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Sep 1 16:18:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:55.706292 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53457ea2-df5b-4208-8163-e26b3551a5ec None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:55.713442 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ff441-64fd-491a-8c7c-39c862bf0933 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:55.759148 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3319e6f-609d-462c-bd7d-dd3637976862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:55.760151 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3319e6f-609d-462c-bd7d-dd3637976862 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:55.760631 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3319e6f-609d-462c-bd7d-dd3637976862 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:55.761094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3319e6f-609d-462c-bd7d-dd3637976862 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:55.762020 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3319e6f-609d-462c-bd7d-dd3637976862 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:55.763749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3319e6f-609d-462c-bd7d-dd3637976862 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:55.766614 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: WARNING keystone.server.flask.common [None req-f3319e6f-609d-462c-bd7d-dd3637976862 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 01 16:18:55.786426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.sql.core [None req-f3319e6f-609d-462c-bd7d-dd3637976862 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 16:18:55.786426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 16:18:55.786426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [parameters: {'id': '8e0cd36592394920b4a8573c05ac6465', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 16:18:55.786426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81503) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 16:18:55.810762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-f3319e6f-609d-462c-bd7d-dd3637976862 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8e0cd36592394920b4a8573c05ac6465.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8e0cd36592394920b4a8573c05ac6465. Sep 01 16:18:55.810762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:55.810762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 16:18:55.810762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 16:18:55.810762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 16:18:55.810762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 16:18:55.810762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 16:18:55.810762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 16:18:55.810762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 16:18:55.810762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application conn.query(q) Sep 01 16:18:55.810762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 16:18:55.810762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 16:18:55.810762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 16:18:55.811820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result.read() Sep 01 16:18:55.811820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 16:18:55.811820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 16:18:55.811820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 16:18:55.811820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 16:18:55.811820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 16:18:55.811820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 16:18:55.811820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 16:18:55.811820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 16:18:55.811820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 16:18:55.811820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:55.811820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 16:18:55.811820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:55.811820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:55.811820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 16:18:55.811820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 16:18:55.811820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 01 16:18:55.811820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 01 16:18:55.812807 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 16:18:55.812807 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application next(self.gen) Sep 01 16:18:55.812807 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 16:18:55.812807 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application yield resource Sep 01 16:18:55.812807 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 16:18:55.812807 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application next(self.gen) Sep 01 16:18:55.812807 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 16:18:55.812807 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 16:18:55.812807 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 16:18:55.812807 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 16:18:55.812807 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 16:18:55.812807 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise self.value Sep 01 16:18:55.812807 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 16:18:55.812807 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 16:18:55.812807 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 16:18:55.812807 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application session.commit() Sep 01 16:18:55.812807 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 16:18:55.813765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 16:18:55.813765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 16:18:55.813765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 16:18:55.813765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 16:18:55.813765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.session.flush() Sep 01 16:18:55.813765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 16:18:55.813765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 16:18:55.813765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 16:18:55.813765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 16:18:55.813765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 16:18:55.813765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application compat.raise_( Sep 01 16:18:55.813765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 16:18:55.813765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception Sep 01 16:18:55.813765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 16:18:55.813765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 16:18:55.813765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 16:18:55.813765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 16:18:55.814737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 16:18:55.814737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 16:18:55.814737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 16:18:55.814737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 16:18:55.814737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 16:18:55.814737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 16:18:55.814737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 16:18:55.814737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 16:18:55.814737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 16:18:55.814737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 16:18:55.814737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 16:18:55.814737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 16:18:55.814737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 16:18:55.814737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 16:18:55.814737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 16:18:55.817914 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 16:18:55.817914 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 16:18:55.817914 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception Sep 01 16:18:55.817914 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 16:18:55.817914 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 16:18:55.817914 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 16:18:55.817914 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 16:18:55.817914 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 16:18:55.817914 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 16:18:55.817914 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 16:18:55.817914 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application conn.query(q) Sep 01 16:18:55.817914 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 16:18:55.817914 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 16:18:55.817914 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 16:18:55.817914 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result.read() Sep 01 16:18:55.817914 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 16:18:55.817914 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 16:18:55.820293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 16:18:55.820293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 16:18:55.820293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 16:18:55.820293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 16:18:55.820293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 16:18:55.820293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 16:18:55.820293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 16:18:55.820293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 16:18:55.820293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [parameters: {'id': '8e0cd36592394920b4a8573c05ac6465', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 16:18:55.820293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 16:18:55.820293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:55.820293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:18:55.820293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:55.820293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:55.820293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 01 16:18:55.821622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 01 16:18:55.821622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 16:18:55.821622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 16:18:55.821622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 01 16:18:55.821622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:55.821622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:18:55.821622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:55.821622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:55.821622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:55.821622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:55.821622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:55.821622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:55.821622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:55.821622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:55.821622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:55.821622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:55.821622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:55.822984 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:55.822984 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 01 16:18:55.822984 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 01 16:18:55.822984 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:55.822984 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:55.822984 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 01 16:18:55.822984 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 01 16:18:55.822984 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8e0cd36592394920b4a8573c05ac6465. Sep 01 16:18:55.822984 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:55.822984 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 464/943] 10.222.0.58 () {60 vars in 1140 bytes} [Thu Sep 1 16:18:55 2022] POST /identity/v3/projects => generated 262 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:18:55.828392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3319e6f-609d-462c-bd7d-dd3637976862 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:55.839105 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53457ea2-df5b-4208-8163-e26b3551a5ec None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:55.868926 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:55.877387 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7484b47-de08-4e3d-828e-46fb8bd821dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:55.878188 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7484b47-de08-4e3d-828e-46fb8bd821dd None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:55.878605 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7484b47-de08-4e3d-828e-46fb8bd821dd None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:55.879016 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7484b47-de08-4e3d-828e-46fb8bd821dd None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:55.879753 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7484b47-de08-4e3d-828e-46fb8bd821dd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:55.880225 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7484b47-de08-4e3d-828e-46fb8bd821dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:55.882747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7484b47-de08-4e3d-828e-46fb8bd821dd None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:55.886621 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:55.893851 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 480/944] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Sep 1 16:18:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:55.895061 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7484b47-de08-4e3d-828e-46fb8bd821dd None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:55.959233 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3319e6f-609d-462c-bd7d-dd3637976862 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:55.996005 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7219b6-fe8b-4d0a-8952-522ae63d8646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:55.996636 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7219b6-fe8b-4d0a-8952-522ae63d8646 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:55.996914 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7219b6-fe8b-4d0a-8952-522ae63d8646 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:55.997186 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7219b6-fe8b-4d0a-8952-522ae63d8646 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:56.011452 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7219b6-fe8b-4d0a-8952-522ae63d8646 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:56.013897 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7219b6-fe8b-4d0a-8952-522ae63d8646 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:56.029220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-ca7219b6-fe8b-4d0a-8952-522ae63d8646 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:56.029220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:56.029220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:56.029220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:56.029220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:56.029220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:56.029220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:56.029220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:56.029220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:56.029220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:56.029220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:56.029220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:56.029220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:56.029220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:56.029220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:56.030145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:56.030145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:56.030145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:56.030145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:56.030145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:56.030145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:56.030145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:56.030145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:56.030145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:56.030145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:56.030145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:56.030145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:56.030145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:56.030145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:56.030145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:56.030145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:56.030145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:56.031120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:56.031120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:56.031120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:56.031120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:56.031120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:56.031120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:56.031120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:56.031120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:56.031566 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7484b47-de08-4e3d-828e-46fb8bd821dd None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:56.033826 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 465/945] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:56.042709 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7219b6-fe8b-4d0a-8952-522ae63d8646 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:56.083835 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00467962-0d14-4650-bc75-54b4ffc00736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:56.085116 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00467962-0d14-4650-bc75-54b4ffc00736 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:56.085401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00467962-0d14-4650-bc75-54b4ffc00736 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:56.085918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00467962-0d14-4650-bc75-54b4ffc00736 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:56.087059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00467962-0d14-4650-bc75-54b4ffc00736 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:56.087634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00467962-0d14-4650-bc75-54b4ffc00736 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:56.090381 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00467962-0d14-4650-bc75-54b4ffc00736 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:56.103880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 481/946] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Sep 1 16:18:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:56.105321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00467962-0d14-4650-bc75-54b4ffc00736 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:56.166271 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:56.186261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7219b6-fe8b-4d0a-8952-522ae63d8646 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:56.195408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:56.232810 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd6c82-ff2a-4503-a21c-297822a6c60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:56.233870 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd6c82-ff2a-4503-a21c-297822a6c60f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:56.234287 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd6c82-ff2a-4503-a21c-297822a6c60f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:56.235036 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd6c82-ff2a-4503-a21c-297822a6c60f None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:56.236070 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd6c82-ff2a-4503-a21c-297822a6c60f None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:56.236453 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd6c82-ff2a-4503-a21c-297822a6c60f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:56.238951 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd6c82-ff2a-4503-a21c-297822a6c60f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:56.244656 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00467962-0d14-4650-bc75-54b4ffc00736 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:56.270475 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 466/947] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Sep 1 16:18:56 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:18:56.286876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ebe16-db1c-474b-b579-6e11e392be23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:56.288242 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ebe16-db1c-474b-b579-6e11e392be23 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:56.288522 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ebe16-db1c-474b-b579-6e11e392be23 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:56.288792 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ebe16-db1c-474b-b579-6e11e392be23 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:56.289579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ebe16-db1c-474b-b579-6e11e392be23 None admin] RBAC: Adding query filter params (interface=public, service_id=959bb408dc334206bcbf716253d8ff74, region_id=RegionOne) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:56.289934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ebe16-db1c-474b-b579-6e11e392be23 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:56.291645 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ebe16-db1c-474b-b579-6e11e392be23 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:56.298492 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 482/948] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Sep 1 16:18:56 2022] GET /identity/v3/endpoints?service_id=959bb408dc334206bcbf716253d8ff74&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:56.303765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd6c82-ff2a-4503-a21c-297822a6c60f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:56.335583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24c24b-23f7-4211-88a6-72a1d54cd885 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:56.335910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24c24b-23f7-4211-88a6-72a1d54cd885 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:56.336124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24c24b-23f7-4211-88a6-72a1d54cd885 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:56.337612 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 483/949] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:18:56.342981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2392b07-a937-49e7-8a0e-8f3a76f428b0 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:56.343299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2392b07-a937-49e7-8a0e-8f3a76f428b0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:56.343534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2392b07-a937-49e7-8a0e-8f3a76f428b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:56.363493 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a2392b07-a937-49e7-8a0e-8f3a76f428b0 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:56.364760 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a2392b07-a937-49e7-8a0e-8f3a76f428b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:56.425321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd6c82-ff2a-4503-a21c-297822a6c60f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:56.441744 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 484/950] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:18:56.466180 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3eeeb-3427-459f-ab30-f6b8a16a5f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:56.467244 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3eeeb-3427-459f-ab30-f6b8a16a5f8f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:56.467735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3eeeb-3427-459f-ab30-f6b8a16a5f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:56.468222 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3eeeb-3427-459f-ab30-f6b8a16a5f8f None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:56.469049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3eeeb-3427-459f-ab30-f6b8a16a5f8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:56.470749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3eeeb-3427-459f-ab30-f6b8a16a5f8f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:56.478182 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 467/951] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Sep 1 16:18:56 2022] GET /identity/v3/services => generated 1497 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:18:56.720719 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2392b07-a937-49e7-8a0e-8f3a76f428b0 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:56.779775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:56.830464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2392b07-a937-49e7-8a0e-8f3a76f428b0 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:56.871787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225f173-c5ef-4e6e-b16a-f2012e88a8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:56.872383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225f173-c5ef-4e6e-b16a-f2012e88a8ed None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:56.872660 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225f173-c5ef-4e6e-b16a-f2012e88a8ed None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:56.872926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225f173-c5ef-4e6e-b16a-f2012e88a8ed None admin] PATH_INFO: `/v3/services/compute` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:56.873510 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225f173-c5ef-4e6e-b16a-f2012e88a8ed None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:56.876274 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225f173-c5ef-4e6e-b16a-f2012e88a8ed None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:56.884845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-4225f173-c5ef-4e6e-b16a-f2012e88a8ed None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:18:56.884845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:56.884845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:18:56.884845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:18:56.884845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:18:56.884845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:18:56.884845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:18:56.884845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:56.884845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:18:56.884845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:56.884845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:18:56.884845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:56.884845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:56.884845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:56.884845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:56.884845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:56.884845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:56.886810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:56.886810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:56.886810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:56.886810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:56.886810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:56.886810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:56.886810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:18:56.886810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:18:56.886810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:18:56.886810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:18:56.886810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:56.886810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:56.886810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:56.886810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:56.886810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:56.886810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:56.887679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:56.887679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:56.887679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:56.887679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:56.887679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:56.887679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:56.887679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:56.887679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:56.887679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:56.887679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:56.887679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:18:56.887679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:56.887679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:18:56.887679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:56.887679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 485/952] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:56.897929 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3eeeb-3427-459f-ab30-f6b8a16a5f8f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:56.928734 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:57.019658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3eeeb-3427-459f-ab30-f6b8a16a5f8f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:57.064217 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82df6b-e46a-4e74-ad08-3f05c404b684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:57.065145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82df6b-e46a-4e74-ad08-3f05c404b684 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:57.065639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82df6b-e46a-4e74-ad08-3f05c404b684 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:57.066110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82df6b-e46a-4e74-ad08-3f05c404b684 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:57.066982 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82df6b-e46a-4e74-ad08-3f05c404b684 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:57.067488 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82df6b-e46a-4e74-ad08-3f05c404b684 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:57.069658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82df6b-e46a-4e74-ad08-3f05c404b684 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:57.075093 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 468/953] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Sep 1 16:18:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:57.080823 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225f173-c5ef-4e6e-b16a-f2012e88a8ed None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:57.186325 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225f173-c5ef-4e6e-b16a-f2012e88a8ed None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:57.203754 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:57.221257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9600ac-e9ee-4a72-b400-41e2a5ac36de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:57.221868 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9600ac-e9ee-4a72-b400-41e2a5ac36de None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:57.222236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9600ac-e9ee-4a72-b400-41e2a5ac36de None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:57.222478 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9600ac-e9ee-4a72-b400-41e2a5ac36de None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:57.223067 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9600ac-e9ee-4a72-b400-41e2a5ac36de None admin] RBAC: Adding query filter params (type=compute) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:57.223366 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9600ac-e9ee-4a72-b400-41e2a5ac36de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:57.224717 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9600ac-e9ee-4a72-b400-41e2a5ac36de None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:57.229876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 486/954] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Sep 1 16:18:57 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:57.315283 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f129e-4936-429d-b22f-b6788ba975cb None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:57.316216 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f129e-4936-429d-b22f-b6788ba975cb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:57.316726 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f129e-4936-429d-b22f-b6788ba975cb None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:57.318675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 469/955] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:57.323993 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe82650-7b1c-404c-93be-82c929d7344a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:57.324542 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe82650-7b1c-404c-93be-82c929d7344a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:57.325056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe82650-7b1c-404c-93be-82c929d7344a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:57.344939 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-1fe82650-7b1c-404c-93be-82c929d7344a None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:57.346485 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1fe82650-7b1c-404c-93be-82c929d7344a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:57.419916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 487/956] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:18:57.690498 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f129e-4936-429d-b22f-b6788ba975cb None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:57.767324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:57.794542 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f129e-4936-429d-b22f-b6788ba975cb None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:57.829133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2f1db-f827-4f45-bc85-e7a789f3de74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:57.829762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2f1db-f827-4f45-bc85-e7a789f3de74 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:57.830040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2f1db-f827-4f45-bc85-e7a789f3de74 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:57.830333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2f1db-f827-4f45-bc85-e7a789f3de74 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:57.840649 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b2f1db-f827-4f45-bc85-e7a789f3de74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:57.845682 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-57b2f1db-f827-4f45-bc85-e7a789f3de74 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:18:57.845682 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:57.845682 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:57.845682 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:57.845682 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:57.845682 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:57.845682 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:57.845682 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:57.845682 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:57.845682 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:57.845682 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:57.845682 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:57.845682 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:57.845682 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:57.845682 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:57.845682 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:57.846748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:57.846748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:57.846748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:57.846748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:57.846748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:57.846748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:57.846748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:57.846748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:57.846748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:57.846748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:57.846748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:57.846748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:57.846748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:57.846748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:57.846748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:57.846748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:57.847678 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:57.847678 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:57.847678 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:57.847678 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:57.847678 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:57.847678 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:57.847678 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:57.847678 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:57.847678 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:57.847678 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:57.847678 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:18:57.847678 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:57.851772 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 470/957] 10.222.0.58 () {56 vars in 1120 bytes} [Thu Sep 1 16:18:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:57.853128 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe82650-7b1c-404c-93be-82c929d7344a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:57.955561 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe82650-7b1c-404c-93be-82c929d7344a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:57.969706 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:57.992674 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f12a13-bf48-4d07-aff5-4dd2878926e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:57.993281 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f12a13-bf48-4d07-aff5-4dd2878926e0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:57.993561 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f12a13-bf48-4d07-aff5-4dd2878926e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:57.993836 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f12a13-bf48-4d07-aff5-4dd2878926e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:57.994541 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f12a13-bf48-4d07-aff5-4dd2878926e0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:57.994848 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f12a13-bf48-4d07-aff5-4dd2878926e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:57.996215 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f12a13-bf48-4d07-aff5-4dd2878926e0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:58.004918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 488/958] 10.222.0.58 () {56 vars in 1115 bytes} [Thu Sep 1 16:18:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:58.011512 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2f1db-f827-4f45-bc85-e7a789f3de74 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:58.122531 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2f1db-f827-4f45-bc85-e7a789f3de74 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:58.157902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ad348-9d7a-4b43-b618-4cc3bf0d8ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:58.158562 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ad348-9d7a-4b43-b618-4cc3bf0d8ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:58.158841 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ad348-9d7a-4b43-b618-4cc3bf0d8ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:58.159113 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ad348-9d7a-4b43-b618-4cc3bf0d8ff9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:58.160990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:58.169728 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1ad348-9d7a-4b43-b618-4cc3bf0d8ff9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:58.171587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1ad348-9d7a-4b43-b618-4cc3bf0d8ff9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:58.190305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-af1ad348-9d7a-4b43-b618-4cc3bf0d8ff9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:58.190305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:58.190305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:58.190305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:58.190305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:58.190305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:58.190305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:58.190305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:58.190305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:58.190305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:58.190305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:58.190305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:58.190305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:58.190305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:58.190305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:58.190305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:58.191305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:58.191305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:58.191305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:58.191305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:58.191305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:58.191305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:58.191305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:58.191305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:58.191305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:58.191305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:58.191305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:58.191305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:58.191305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:58.191305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:58.191305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:58.191305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:58.191305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:58.192230 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:58.192230 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:58.192230 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:58.192230 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 471/959] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:58.199250 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f12a13-bf48-4d07-aff5-4dd2878926e0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:58.309923 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5e72e-fbed-475d-b6f1-6c0ac3f3e833 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:58.310243 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5e72e-fbed-475d-b6f1-6c0ac3f3e833 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:58.310422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5e72e-fbed-475d-b6f1-6c0ac3f3e833 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:58.311903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 472/960] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:58.317346 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7e147-a00d-493c-8679-f690ab67871e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:58.318033 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f12a13-bf48-4d07-aff5-4dd2878926e0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:58.319361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7e147-a00d-493c-8679-f690ab67871e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:58.319840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7e147-a00d-493c-8679-f690ab67871e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:58.343191 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7ae7e147-a00d-493c-8679-f690ab67871e None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:58.345266 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7ae7e147-a00d-493c-8679-f690ab67871e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:58.356892 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7126a-0378-4ffe-9250-2c17e2a1f6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:58.357493 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7126a-0378-4ffe-9250-2c17e2a1f6b9 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:58.357769 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7126a-0378-4ffe-9250-2c17e2a1f6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:58.358038 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7126a-0378-4ffe-9250-2c17e2a1f6b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:58.358739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7126a-0378-4ffe-9250-2c17e2a1f6b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:58.359009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7126a-0378-4ffe-9250-2c17e2a1f6b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:58.360366 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7126a-0378-4ffe-9250-2c17e2a1f6b9 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:58.377737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 489/961] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:58.379086 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7126a-0378-4ffe-9250-2c17e2a1f6b9 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:58.415283 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:58.446236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 473/962] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:18:58.474490 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a263aef-1e5f-4ee7-b6ce-2c3ae68fdac6 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:58.474846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a263aef-1e5f-4ee7-b6ce-2c3ae68fdac6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:58.475100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a263aef-1e5f-4ee7-b6ce-2c3ae68fdac6 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:58.476723 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 474/963] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:58.481940 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9a6c4-add9-4f86-ab1c-188ebea77171 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:58.482255 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9a6c4-add9-4f86-ab1c-188ebea77171 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:58.482544 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9a6c4-add9-4f86-ab1c-188ebea77171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:58.501420 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-45c9a6c4-add9-4f86-ab1c-188ebea77171 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:58.502698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-45c9a6c4-add9-4f86-ab1c-188ebea77171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:58.506984 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7126a-0378-4ffe-9250-2c17e2a1f6b9 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:58.554206 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836f57f-186c-4829-916c-23f3cb6a7f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:58.554855 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836f57f-186c-4829-916c-23f3cb6a7f0f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:58.555504 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836f57f-186c-4829-916c-23f3cb6a7f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:58.555780 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836f57f-186c-4829-916c-23f3cb6a7f0f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:58.564953 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836f57f-186c-4829-916c-23f3cb6a7f0f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:58.566967 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836f57f-186c-4829-916c-23f3cb6a7f0f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:58.581156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-f836f57f-186c-4829-916c-23f3cb6a7f0f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:18:58.581156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:58.581156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:58.581156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:58.581156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:58.581156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:58.581156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:58.581156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:58.581156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:58.581156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:58.581156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:58.581156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:58.581156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:58.581156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:58.581156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:58.581156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:58.582124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:58.582124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:58.582124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:58.582124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:58.582124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:58.582124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:58.582124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:58.582124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:58.582124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:58.582124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:58.582124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:58.582124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:58.582124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:58.582124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:58.582124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:58.582124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:58.582124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:58.582124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:58.583721 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:58.583721 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:58.583721 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:58.583721 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:58.583721 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:58.583721 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:58.583721 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:58.583721 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:58.583721 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:18:58.583721 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:18:58.583721 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 490/965] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Sep 1 16:18:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:18:58.585196 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 475/965] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:18:58.591111 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836f57f-186c-4829-916c-23f3cb6a7f0f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:58.687312 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:58.694104 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836f57f-186c-4829-916c-23f3cb6a7f0f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:58.729725 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c13523-ec1d-40e0-b360-e0f3480e96a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:58.730355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c13523-ec1d-40e0-b360-e0f3480e96a4 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:58.730634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c13523-ec1d-40e0-b360-e0f3480e96a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:58.730904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c13523-ec1d-40e0-b360-e0f3480e96a4 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:58.731641 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c13523-ec1d-40e0-b360-e0f3480e96a4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:58.731950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c13523-ec1d-40e0-b360-e0f3480e96a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:58.733364 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c13523-ec1d-40e0-b360-e0f3480e96a4 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:58.734593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9a6c4-add9-4f86-ab1c-188ebea77171 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:58.755123 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 491/966] 10.222.0.58 () {56 vars in 1149 bytes} [Thu Sep 1 16:18:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:18:58.761161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c13523-ec1d-40e0-b360-e0f3480e96a4 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:58.789731 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:58.849202 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9a6c4-add9-4f86-ab1c-188ebea77171 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:58.867506 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c13523-ec1d-40e0-b360-e0f3480e96a4 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:58.885691 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70645b4d-ee0a-470d-9003-f42d775cb5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:58.886326 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70645b4d-ee0a-470d-9003-f42d775cb5f6 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:58.886608 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70645b4d-ee0a-470d-9003-f42d775cb5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:58.886878 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70645b4d-ee0a-470d-9003-f42d775cb5f6 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:58.887510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70645b4d-ee0a-470d-9003-f42d775cb5f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:58.888930 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70645b4d-ee0a-470d-9003-f42d775cb5f6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:58.904052 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ff146-9d19-4cfc-b3dc-9bee4793529a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:58.904610 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.sql.core [None req-70645b4d-ee0a-470d-9003-f42d775cb5f6 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 16:18:58.904610 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 16:18:58.904610 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [parameters: {'user_id': 'c4d3905fb14f4e629845eac79ea08038', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 16:18:58.904610 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81503) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 16:18:58.904964 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ff146-9d19-4cfc-b3dc-9bee4793529a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:58.904964 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ff146-9d19-4cfc-b3dc-9bee4793529a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:58.905196 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ff146-9d19-4cfc-b3dc-9bee4793529a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:58.912427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-70645b4d-ee0a-470d-9003-f42d775cb5f6 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 16:18:58.912427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:58.912427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 16:18:58.912427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 16:18:58.912427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 16:18:58.912427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 16:18:58.912427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 16:18:58.912427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 16:18:58.912427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 16:18:58.912427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application conn.query(q) Sep 01 16:18:58.912427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 16:18:58.912427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 16:18:58.912427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 16:18:58.912427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result.read() Sep 01 16:18:58.912427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 16:18:58.913324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 16:18:58.913324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 16:18:58.913324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 16:18:58.913324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 16:18:58.913324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 16:18:58.913324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 16:18:58.913324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 16:18:58.913324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 16:18:58.913324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:58.913324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 16:18:58.913324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:58.913324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:58.913324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 16:18:58.913324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 16:18:58.913324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 01 16:18:58.913324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 01 16:18:58.913324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 16:18:58.913324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application next(self.gen) Sep 01 16:18:58.918657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 16:18:58.918657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application yield resource Sep 01 16:18:58.918657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 16:18:58.918657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application next(self.gen) Sep 01 16:18:58.918657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 16:18:58.918657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 16:18:58.918657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 16:18:58.918657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 16:18:58.918657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 16:18:58.918657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise self.value Sep 01 16:18:58.918657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 16:18:58.918657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 16:18:58.918657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 16:18:58.918657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application session.commit() Sep 01 16:18:58.918657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 16:18:58.918657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 16:18:58.918657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 16:18:58.919608 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 16:18:58.919608 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 16:18:58.919608 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.session.flush() Sep 01 16:18:58.919608 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 16:18:58.919608 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 16:18:58.919608 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 16:18:58.919608 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 16:18:58.919608 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 16:18:58.919608 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application compat.raise_( Sep 01 16:18:58.919608 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 16:18:58.919608 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception Sep 01 16:18:58.919608 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 16:18:58.919608 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 16:18:58.919608 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 16:18:58.919608 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 16:18:58.919608 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 16:18:58.919608 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 16:18:58.920630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 16:18:58.920630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 16:18:58.920630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 01 16:18:58.920630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 01 16:18:58.920630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 16:18:58.920630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 16:18:58.920630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 16:18:58.920630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 16:18:58.920630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 16:18:58.920630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 16:18:58.920630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 16:18:58.920630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 16:18:58.920630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 16:18:58.920630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 16:18:58.920630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 16:18:58.920630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception Sep 01 16:18:58.921597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 16:18:58.921597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 16:18:58.921597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 16:18:58.921597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 16:18:58.921597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 16:18:58.921597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 16:18:58.921597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 16:18:58.921597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application conn.query(q) Sep 01 16:18:58.921597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 16:18:58.921597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 16:18:58.921597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 16:18:58.921597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result.read() Sep 01 16:18:58.921597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 16:18:58.921597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 16:18:58.921597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 16:18:58.921597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 16:18:58.922755 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 16:18:58.922755 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 16:18:58.922755 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 16:18:58.922755 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 16:18:58.922755 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 16:18:58.922755 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 16:18:58.922755 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c4d3905fb14f4e629845eac79ea08038', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 16:18:58.922755 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 16:18:58.922755 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:58.922755 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:18:58.922755 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:58.922755 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:58.922755 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:58.922755 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:58.922755 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:58.923673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:58.923673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:58.923673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:58.923673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:58.923673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:58.923673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:58.923673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:58.923673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 01 16:18:58.923673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 01 16:18:58.923673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:58.923673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:58.923673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:58.923673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:58.923673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:58.923673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:58.923673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 01 16:18:58.923673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 01 16:18:58.924720 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 01 16:18:58.924720 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 01 16:18:58.924720 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 16:18:58.924720 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 16:18:58.924720 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 16:18:58.924720 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:58.924720 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 476/967] 10.222.0.58 () {60 vars in 1131 bytes} [Thu Sep 1 16:18:58 2022] POST /identity/v3/users => generated 160 bytes in 180 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:58.924720 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70645b4d-ee0a-470d-9003-f42d775cb5f6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:58.935851 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:58.984150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2ff146-9d19-4cfc-b3dc-9bee4793529a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:58.985574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2ff146-9d19-4cfc-b3dc-9bee4793529a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:59.031508 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70645b4d-ee0a-470d-9003-f42d775cb5f6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:59.042552 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:59.072219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c88919d-04cc-4755-bfec-f71afffa6745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:59.072862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88919d-04cc-4755-bfec-f71afffa6745 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:59.073144 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88919d-04cc-4755-bfec-f71afffa6745 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:59.073417 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88919d-04cc-4755-bfec-f71afffa6745 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:59.074009 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c88919d-04cc-4755-bfec-f71afffa6745 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:59.075637 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c88919d-04cc-4755-bfec-f71afffa6745 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:59.085210 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-6c88919d-04cc-4755-bfec-f71afffa6745 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:18:59.085210 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:59.085210 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:18:59.085210 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:18:59.085210 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:18:59.085210 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:18:59.085210 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:18:59.085210 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:59.085210 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:18:59.085210 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:59.085210 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:18:59.085210 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:59.085210 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:59.085210 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:59.085210 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:59.085210 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:59.085210 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:59.086345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:59.086345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:59.086345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:59.086345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:59.086345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:59.086345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:59.086345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:18:59.086345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:18:59.086345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:18:59.086345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:18:59.086345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:59.086345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:59.086345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:59.086345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:59.086345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:59.086345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:59.087226 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:59.087226 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:59.087226 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:59.087226 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:59.087226 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:59.087226 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:59.087226 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:59.087226 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:59.087226 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:59.087226 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:59.087226 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:18:59.087226 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:59.087226 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:18:59.087226 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:59.087226 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 477/968] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Sep 1 16:18:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:59.088052 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c88919d-04cc-4755-bfec-f71afffa6745 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:59.109061 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 492/969] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:18:58 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 16:18:59.110309 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ff146-9d19-4cfc-b3dc-9bee4793529a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:59.165146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:59.200607 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c88919d-04cc-4755-bfec-f71afffa6745 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:59.216214 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ff146-9d19-4cfc-b3dc-9bee4793529a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:59.235875 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936206e2-e5cb-4214-91d7-ff6ab056bbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:59.236515 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936206e2-e5cb-4214-91d7-ff6ab056bbaf None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:59.236791 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936206e2-e5cb-4214-91d7-ff6ab056bbaf None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:59.237071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936206e2-e5cb-4214-91d7-ff6ab056bbaf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:59.245846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936206e2-e5cb-4214-91d7-ff6ab056bbaf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:59.247310 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936206e2-e5cb-4214-91d7-ff6ab056bbaf None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:59.253964 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af94d1-459b-4b77-9e2d-1be9cea27120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:59.254828 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af94d1-459b-4b77-9e2d-1be9cea27120 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:59.255303 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af94d1-459b-4b77-9e2d-1be9cea27120 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:59.255752 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af94d1-459b-4b77-9e2d-1be9cea27120 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:59.257108 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af94d1-459b-4b77-9e2d-1be9cea27120 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:59.257598 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af94d1-459b-4b77-9e2d-1be9cea27120 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:59.259001 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-936206e2-e5cb-4214-91d7-ff6ab056bbaf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:18:59.259001 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:59.259001 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:59.259001 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:59.259001 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:59.259001 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:59.259001 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:59.259001 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:59.259001 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:59.259001 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:59.259001 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:59.259001 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:59.259001 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:59.259001 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:59.259001 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:59.259001 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:59.259961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:59.259961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:59.259961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:59.259961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:59.259961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:59.259961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:59.259961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:59.259961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:59.259961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:59.259961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:59.259961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:59.259961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:59.259961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:59.259961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:59.259961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:59.259961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:59.259961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:59.259961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:59.260939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:59.260939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:59.260939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:59.260939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:59.260939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:59.260939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:59.260939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:59.260939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:59.260939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:18:59.260939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:59.260939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 478/970] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:18:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:59.260939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936206e2-e5cb-4214-91d7-ff6ab056bbaf None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:59.262022 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af94d1-459b-4b77-9e2d-1be9cea27120 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:59.269256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 493/971] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Sep 1 16:18:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:59.270325 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af94d1-459b-4b77-9e2d-1be9cea27120 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:59.300210 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:59.370397 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936206e2-e5cb-4214-91d7-ff6ab056bbaf None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:59.373759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af94d1-459b-4b77-9e2d-1be9cea27120 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:59.389524 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:59.403868 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a13cc6-0df7-482a-b6e3-062bede25f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:59.404498 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a13cc6-0df7-482a-b6e3-062bede25f84 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:59.404779 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a13cc6-0df7-482a-b6e3-062bede25f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:59.405050 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a13cc6-0df7-482a-b6e3-062bede25f84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:59.407539 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ecb3b1-fc6e-479c-ba4b-94165c3d01e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:59.408136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecb3b1-fc6e-479c-ba4b-94165c3d01e9 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:59.408413 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecb3b1-fc6e-479c-ba4b-94165c3d01e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:59.408682 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecb3b1-fc6e-479c-ba4b-94165c3d01e9 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:59.409380 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ecb3b1-fc6e-479c-ba4b-94165c3d01e9 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:59.409679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ecb3b1-fc6e-479c-ba4b-94165c3d01e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:59.411130 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ecb3b1-fc6e-479c-ba4b-94165c3d01e9 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:59.415864 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a13cc6-0df7-482a-b6e3-062bede25f84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:59.417549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a13cc6-0df7-482a-b6e3-062bede25f84 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:59.431359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-f0a13cc6-0df7-482a-b6e3-062bede25f84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:59.431359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:59.431359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:59.431359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:59.431359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:59.431359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:59.431359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:59.431359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:59.431359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:59.431359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:59.431359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:59.431359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:59.431359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:59.431359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:59.431359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:59.431359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:59.438045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:59.438045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:59.438045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:59.438045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:59.438045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:59.438045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:59.438045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:59.438045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:59.438045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:59.438045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:59.438045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:59.438045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:59.438045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:59.438045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:59.438045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:59.438045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:59.438045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:59.439241 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:59.439241 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:59.439241 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:59.439241 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 479/972] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:59.439241 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a13cc6-0df7-482a-b6e3-062bede25f84 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:59.452204 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 494/973] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Sep 1 16:18:59 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:59.453460 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd41e8-5b44-43ce-90c2-53e9e7eb1213 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:59.453744 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd41e8-5b44-43ce-90c2-53e9e7eb1213 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:59.454031 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd41e8-5b44-43ce-90c2-53e9e7eb1213 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:59.455416 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 495/974] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:59.459787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd41e8-5b44-43ce-90c2-53e9e7eb1213 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:59.530612 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:59.549005 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a13cc6-0df7-482a-b6e3-062bede25f84 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:59.585642 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd41e8-5b44-43ce-90c2-53e9e7eb1213 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:59.589494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d1365-d7bf-4388-a5f2-fc19a6274ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:59.590650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d1365-d7bf-4388-a5f2-fc19a6274ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:59.590650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d1365-d7bf-4388-a5f2-fc19a6274ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:59.590839 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d1365-d7bf-4388-a5f2-fc19a6274ed5 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:59.591330 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61d1365-d7bf-4388-a5f2-fc19a6274ed5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:59.591639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61d1365-d7bf-4388-a5f2-fc19a6274ed5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:59.593020 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61d1365-d7bf-4388-a5f2-fc19a6274ed5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:59.604091 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 480/975] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Sep 1 16:18:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:59.605629 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bdadd-9a70-46c8-8386-d77def852abe None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:59.605629 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bdadd-9a70-46c8-8386-d77def852abe None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:59.605921 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bdadd-9a70-46c8-8386-d77def852abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:59.624835 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661fe26-8ea6-4a76-934b-9e2f3a8c6259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:59.625337 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661fe26-8ea6-4a76-934b-9e2f3a8c6259 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:59.625588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661fe26-8ea6-4a76-934b-9e2f3a8c6259 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:59.625863 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661fe26-8ea6-4a76-934b-9e2f3a8c6259 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:59.626600 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661fe26-8ea6-4a76-934b-9e2f3a8c6259 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:59.626902 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661fe26-8ea6-4a76-934b-9e2f3a8c6259 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:59.628253 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661fe26-8ea6-4a76-934b-9e2f3a8c6259 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:59.628369 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6a3bdadd-9a70-46c8-8386-d77def852abe None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:59.629859 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6a3bdadd-9a70-46c8-8386-d77def852abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:59.641717 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 496/976] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:59.642577 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661fe26-8ea6-4a76-934b-9e2f3a8c6259 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:59.684701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:59.711130 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 481/977] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:18:59.711804 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3bdadd-9a70-46c8-8386-d77def852abe None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:59.760270 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661fe26-8ea6-4a76-934b-9e2f3a8c6259 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:59.793990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:59.795288 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bdf227-82e1-4402-8a3e-4b1f13f82a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:59.796032 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bdf227-82e1-4402-8a3e-4b1f13f82a2f None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:59.796383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bdf227-82e1-4402-8a3e-4b1f13f82a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:59.796685 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bdf227-82e1-4402-8a3e-4b1f13f82a2f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:59.797348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bdf227-82e1-4402-8a3e-4b1f13f82a2f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:59.798741 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bdf227-82e1-4402-8a3e-4b1f13f82a2f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:59.818578 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3bdadd-9a70-46c8-8386-d77def852abe None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:59.820820 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 497/978] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:18:59 2022] POST /identity/v3/endpoints => generated 349 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:18:59.863339 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d01fe41-896e-47bb-9dc1-5267c38f2d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:59.863991 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d01fe41-896e-47bb-9dc1-5267c38f2d22 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:59.864271 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d01fe41-896e-47bb-9dc1-5267c38f2d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:59.864543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d01fe41-896e-47bb-9dc1-5267c38f2d22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:59.873035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d01fe41-896e-47bb-9dc1-5267c38f2d22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:59.874541 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d01fe41-896e-47bb-9dc1-5267c38f2d22 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:59.886978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-0d01fe41-896e-47bb-9dc1-5267c38f2d22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:59.886978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:59.886978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:59.886978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:59.886978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:59.886978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:59.886978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:59.886978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:59.886978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:59.886978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:59.886978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:59.886978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:59.886978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:59.886978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:59.886978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:59.888014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:59.888014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:59.888014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:59.888014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:59.888014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:59.888014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:59.888014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:59.888014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:59.888014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:59.888014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:59.888014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:59.888014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:59.888014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:59.888014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:59.888014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:59.888014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:59.888014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:59.888936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:59.888936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:59.888936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:59.888936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:59.888936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:59.888936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:59.888936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:59.888936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:18:59.890526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 482/979] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:59.893888 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bdf227-82e1-4402-8a3e-4b1f13f82a2f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:59.980332 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d01fe41-896e-47bb-9dc1-5267c38f2d22 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:59.994377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:00.006750 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bdf227-82e1-4402-8a3e-4b1f13f82a2f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:00.041139 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3294d891-00aa-4175-a503-cd3dd7520b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:00.042179 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294d891-00aa-4175-a503-cd3dd7520b80 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:00.045526 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294d891-00aa-4175-a503-cd3dd7520b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:00.045526 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294d891-00aa-4175-a503-cd3dd7520b80 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:00.045526 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3294d891-00aa-4175-a503-cd3dd7520b80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:00.045526 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3294d891-00aa-4175-a503-cd3dd7520b80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:00.045526 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3294d891-00aa-4175-a503-cd3dd7520b80 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:00.062652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 498/980] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:00.064767 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3294d891-00aa-4175-a503-cd3dd7520b80 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:00.107632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d01fe41-896e-47bb-9dc1-5267c38f2d22 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:00.144432 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cbb9ee-ead4-419f-90e8-da8c63ed0292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:00.144958 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbb9ee-ead4-419f-90e8-da8c63ed0292 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:00.145414 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbb9ee-ead4-419f-90e8-da8c63ed0292 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:00.145538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbb9ee-ead4-419f-90e8-da8c63ed0292 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:00.146136 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cbb9ee-ead4-419f-90e8-da8c63ed0292 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:00.147547 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cbb9ee-ead4-419f-90e8-da8c63ed0292 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:00.151010 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:00.165572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 483/981] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Sep 1 16:18:59 2022] POST /identity/v3/services => generated 242 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:19:00.169139 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3294d891-00aa-4175-a503-cd3dd7520b80 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:00.217375 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570573a9-e944-4216-bd9c-123e1b683a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:00.218006 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570573a9-e944-4216-bd9c-123e1b683a1d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:00.218314 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570573a9-e944-4216-bd9c-123e1b683a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:00.218588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570573a9-e944-4216-bd9c-123e1b683a1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:00.219327 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570573a9-e944-4216-bd9c-123e1b683a1d None admin] RBAC: Adding query filter params (role.id=1ff98ccdce644654b43e48dc28fce36b, scope.project.id=479530ced580426b85d1fc07af43417f, user.id=dd7b98536fa445cebd552fcbf54b757d) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:00.219820 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570573a9-e944-4216-bd9c-123e1b683a1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:00.221085 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570573a9-e944-4216-bd9c-123e1b683a1d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:00.228047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 499/982] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:19:00 2022] GET /identity/v3/role_assignments?user.id=dd7b98536fa445cebd552fcbf54b757d&scope.project.id=479530ced580426b85d1fc07af43417f&role.id=1ff98ccdce644654b43e48dc28fce36b => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:01.737822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03654e-7b04-474d-88fc-cde07c10a332 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:01.738416 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03654e-7b04-474d-88fc-cde07c10a332 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:01.738416 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03654e-7b04-474d-88fc-cde07c10a332 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:01.739872 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 484/983] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:19:01.745286 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe1b36-b5a6-445f-ac58-3d0cea9e0707 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:01.745667 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe1b36-b5a6-445f-ac58-3d0cea9e0707 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:01.745848 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe1b36-b5a6-445f-ac58-3d0cea9e0707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:01.764345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-23fe1b36-b5a6-445f-ac58-3d0cea9e0707 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:01.765518 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-23fe1b36-b5a6-445f-ac58-3d0cea9e0707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:01.768410 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:01.838365 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 500/984] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:19:02.105571 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03654e-7b04-474d-88fc-cde07c10a332 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:02.108441 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180ee57-7034-4e29-ac4e-07cbdead58ed None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:02.108724 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180ee57-7034-4e29-ac4e-07cbdead58ed None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:02.108998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180ee57-7034-4e29-ac4e-07cbdead58ed None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:02.110367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 501/985] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:19:02.116022 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c0154-ee94-4d16-8edb-85a23f849e04 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:02.116301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c0154-ee94-4d16-8edb-85a23f849e04 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:02.116574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c0154-ee94-4d16-8edb-85a23f849e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:02.135408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f41c0154-ee94-4d16-8edb-85a23f849e04 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:02.136659 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f41c0154-ee94-4d16-8edb-85a23f849e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:02.208151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 502/986] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:19:02.209410 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7345c-31cb-4151-b6be-2d5927535c54 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:02.209690 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7345c-31cb-4151-b6be-2d5927535c54 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:02.209960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7345c-31cb-4151-b6be-2d5927535c54 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:02.214362 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 503/987] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:19:02.214496 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03654e-7b04-474d-88fc-cde07c10a332 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:02.216661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a2fb2-30ea-4ea8-8281-5db3d8157852 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:02.216947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a2fb2-30ea-4ea8-8281-5db3d8157852 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:02.217220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a2fb2-30ea-4ea8-8281-5db3d8157852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:02.235024 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-949a2fb2-30ea-4ea8-8281-5db3d8157852 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:02.236198 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-949a2fb2-30ea-4ea8-8281-5db3d8157852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:02.249454 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34502bd5-a6b0-48ec-9365-446166a69616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:02.250081 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34502bd5-a6b0-48ec-9365-446166a69616 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:02.250399 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34502bd5-a6b0-48ec-9365-446166a69616 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:02.250677 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34502bd5-a6b0-48ec-9365-446166a69616 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:02.251273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34502bd5-a6b0-48ec-9365-446166a69616 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:02.252633 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34502bd5-a6b0-48ec-9365-446166a69616 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:02.262736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-34502bd5-a6b0-48ec-9365-446166a69616 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:19:02.262736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:02.262736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:19:02.262736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:19:02.262736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:19:02.262736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:19:02.262736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:19:02.262736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:19:02.262736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:19:02.262736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:02.262736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:19:02.262736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:02.262736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:02.262736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:02.262736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:02.262736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:02.262736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:02.263942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:02.263942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:02.263942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:02.263942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:02.263942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:02.263942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:02.263942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:19:02.263942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:19:02.263942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:19:02.263942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:19:02.263942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:02.263942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:02.263942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:02.263942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:02.263942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:02.263942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:02.265006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:02.265006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:02.265006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:02.265006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:02.265006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:02.265006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:02.265006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:02.265006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:02.265006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:02.265006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:02.265006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:19:02.265006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:19:02.265006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:19:02.265006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:02.265006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 485/988] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Sep 1 16:19:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:19:02.270016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34502bd5-a6b0-48ec-9365-446166a69616 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:02.305570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:02.311131 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 504/989] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:19:02.337645 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:02.381854 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34502bd5-a6b0-48ec-9365-446166a69616 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:02.415853 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c125ea-cba1-4400-9543-4d77cb65221b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:02.416453 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c125ea-cba1-4400-9543-4d77cb65221b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:02.416987 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c125ea-cba1-4400-9543-4d77cb65221b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:02.417313 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c125ea-cba1-4400-9543-4d77cb65221b None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:02.417935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c125ea-cba1-4400-9543-4d77cb65221b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:02.418255 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c125ea-cba1-4400-9543-4d77cb65221b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:02.419605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c125ea-cba1-4400-9543-4d77cb65221b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:02.424585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 486/990] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Sep 1 16:19:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:19:02.430680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a2fb2-30ea-4ea8-8281-5db3d8157852 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:02.470632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c125ea-cba1-4400-9543-4d77cb65221b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:02.535976 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a2fb2-30ea-4ea8-8281-5db3d8157852 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:02.573554 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb21fb-7a57-4e35-ae11-a65cb5f10fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:02.574487 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb21fb-7a57-4e35-ae11-a65cb5f10fbd None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:02.574960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb21fb-7a57-4e35-ae11-a65cb5f10fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:02.575236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb21fb-7a57-4e35-ae11-a65cb5f10fbd None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:02.575873 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb21fb-7a57-4e35-ae11-a65cb5f10fbd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:02.576175 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb21fb-7a57-4e35-ae11-a65cb5f10fbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:02.577525 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb21fb-7a57-4e35-ae11-a65cb5f10fbd None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:02.579534 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c125ea-cba1-4400-9543-4d77cb65221b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:02.582473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 505/991] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Sep 1 16:19:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:02.583187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb21fb-7a57-4e35-ae11-a65cb5f10fbd None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:02.604229 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:02.619208 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6edd83-a780-40d6-a2ce-22f6af9fe673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:02.620271 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6edd83-a780-40d6-a2ce-22f6af9fe673 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:02.620764 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6edd83-a780-40d6-a2ce-22f6af9fe673 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:02.621262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6edd83-a780-40d6-a2ce-22f6af9fe673 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:02.622330 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6edd83-a780-40d6-a2ce-22f6af9fe673 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:02.623732 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6edd83-a780-40d6-a2ce-22f6af9fe673 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:02.634153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-ff6edd83-a780-40d6-a2ce-22f6af9fe673 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:19:02.634153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:02.634153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:19:02.634153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:19:02.634153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:19:02.634153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:19:02.634153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:19:02.634153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:19:02.634153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:19:02.634153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:02.634153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:19:02.634153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:02.634153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:02.634153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:02.634153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:02.634153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:02.634153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:02.636536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:02.636536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:02.636536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:02.636536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:02.636536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:02.636536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:02.636536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:19:02.636536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:19:02.636536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:19:02.636536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:19:02.636536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:02.636536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:02.636536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:02.636536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:02.636536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:02.636536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:02.637506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:02.637506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:02.637506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:02.637506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:02.637506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:02.637506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:02.637506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:02.637506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:02.637506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:02.637506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:02.637506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:19:02.637506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:19:02.637506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:19:02.637506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:02.637506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 487/992] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:19:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:19:02.638361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6edd83-a780-40d6-a2ce-22f6af9fe673 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:02.648879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:02.701928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb21fb-7a57-4e35-ae11-a65cb5f10fbd None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:02.735419 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135895b-1b07-4f5d-9450-550bad7c921d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:02.736043 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135895b-1b07-4f5d-9450-550bad7c921d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:02.736322 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135895b-1b07-4f5d-9450-550bad7c921d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:02.736595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135895b-1b07-4f5d-9450-550bad7c921d None admin] PATH_INFO: `/v3/services/image` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:02.737175 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8135895b-1b07-4f5d-9450-550bad7c921d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:02.738563 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8135895b-1b07-4f5d-9450-550bad7c921d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:02.741470 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6edd83-a780-40d6-a2ce-22f6af9fe673 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:02.747661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-8135895b-1b07-4f5d-9450-550bad7c921d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:19:02.747661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:02.747661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:19:02.747661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:19:02.747661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:19:02.747661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:19:02.747661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:19:02.747661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:19:02.747661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:19:02.747661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:19:02.747661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:19:02.747661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:19:02.747661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:02.747661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:02.747661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:02.747661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:02.747661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:02.748910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:02.748910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:02.748910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:02.748910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:02.748910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:02.748910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:02.748910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:19:02.748910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:19:02.748910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:19:02.748910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:19:02.748910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:02.748910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:02.748910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:02.748910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:02.748910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:02.748910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:02.749789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:02.749789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:02.749789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:02.749789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:02.749789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:02.749789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:02.749789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:02.749789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:02.749789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:02.749789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:02.749789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:19:02.749789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:19:02.749789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:19:02.749789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:19:02.749789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 506/993] 10.222.0.58 () {56 vars in 1105 bytes} [Thu Sep 1 16:19:02 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:19:02.750736 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135895b-1b07-4f5d-9450-550bad7c921d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:02.777822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deace29-408f-4be0-a5c8-e402565ab584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:02.778521 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deace29-408f-4be0-a5c8-e402565ab584 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:02.778807 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deace29-408f-4be0-a5c8-e402565ab584 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:02.779089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deace29-408f-4be0-a5c8-e402565ab584 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:02.779801 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deace29-408f-4be0-a5c8-e402565ab584 None admin] RBAC: Adding query filter params (interface=admin, service_id=959bb408dc334206bcbf716253d8ff74, region_id=RegionOne) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:02.780111 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deace29-408f-4be0-a5c8-e402565ab584 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:02.781615 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deace29-408f-4be0-a5c8-e402565ab584 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:02.786583 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 488/994] 10.222.0.58 () {56 vars in 1249 bytes} [Thu Sep 1 16:19:02 2022] GET /identity/v3/endpoints?service_id=959bb408dc334206bcbf716253d8ff74&interface=admin®ion_id=RegionOne => generated 193 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:02.787874 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deace29-408f-4be0-a5c8-e402565ab584 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:02.835831 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:02.847946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135895b-1b07-4f5d-9450-550bad7c921d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:02.874815 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:02.883190 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eeaba3-dc18-47ab-b5f8-5afe4ae7b0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:02.883791 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eeaba3-dc18-47ab-b5f8-5afe4ae7b0d6 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:02.884068 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eeaba3-dc18-47ab-b5f8-5afe4ae7b0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:02.884338 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eeaba3-dc18-47ab-b5f8-5afe4ae7b0d6 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:02.884960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eeaba3-dc18-47ab-b5f8-5afe4ae7b0d6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:02.885258 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eeaba3-dc18-47ab-b5f8-5afe4ae7b0d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:02.886477 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deace29-408f-4be0-a5c8-e402565ab584 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:02.886653 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eeaba3-dc18-47ab-b5f8-5afe4ae7b0d6 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:02.891246 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 507/995] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Sep 1 16:19:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:02.892008 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eeaba3-dc18-47ab-b5f8-5afe4ae7b0d6 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:02.920689 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4433f1-826d-48fe-9d67-b70eeb8cdcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:02.921306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4433f1-826d-48fe-9d67-b70eeb8cdcd6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:02.921568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4433f1-826d-48fe-9d67-b70eeb8cdcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:02.921839 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4433f1-826d-48fe-9d67-b70eeb8cdcd6 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:02.922501 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4433f1-826d-48fe-9d67-b70eeb8cdcd6 None admin] RBAC: Adding query filter params (name=image) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:02.922802 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4433f1-826d-48fe-9d67-b70eeb8cdcd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:02.924151 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4433f1-826d-48fe-9d67-b70eeb8cdcd6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:02.928827 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 489/996] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:19:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:02.929573 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4433f1-826d-48fe-9d67-b70eeb8cdcd6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:02.991318 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eeaba3-dc18-47ab-b5f8-5afe4ae7b0d6 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:03.024809 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b34ef-4150-4706-baf1-7c0c57690985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:03.025413 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b34ef-4150-4706-baf1-7c0c57690985 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:03.025687 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b34ef-4150-4706-baf1-7c0c57690985 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:03.025957 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b34ef-4150-4706-baf1-7c0c57690985 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:03.026583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b34ef-4150-4706-baf1-7c0c57690985 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:03.027933 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b34ef-4150-4706-baf1-7c0c57690985 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:03.033927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 508/997] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Sep 1 16:19:02 2022] GET /identity/v3/services => generated 1727 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:19:03.034736 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b34ef-4150-4706-baf1-7c0c57690985 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:03.036073 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4433f1-826d-48fe-9d67-b70eeb8cdcd6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:03.052935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:03.080289 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11355925-734b-4194-a130-05e968d52e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:03.080918 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11355925-734b-4194-a130-05e968d52e50 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:03.081204 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11355925-734b-4194-a130-05e968d52e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:03.081470 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11355925-734b-4194-a130-05e968d52e50 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:03.082121 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11355925-734b-4194-a130-05e968d52e50 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:03.082456 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11355925-734b-4194-a130-05e968d52e50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:03.083903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11355925-734b-4194-a130-05e968d52e50 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:03.090254 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 490/998] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Sep 1 16:19:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:03.095951 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11355925-734b-4194-a130-05e968d52e50 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:03.116567 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:03.167820 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b34ef-4150-4706-baf1-7c0c57690985 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:03.205210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c45bd-c766-419b-a284-4480dcc8a699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:03.205913 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c45bd-c766-419b-a284-4480dcc8a699 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:03.206295 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c45bd-c766-419b-a284-4480dcc8a699 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:03.206666 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c45bd-c766-419b-a284-4480dcc8a699 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:03.207377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8c45bd-c766-419b-a284-4480dcc8a699 None admin] RBAC: Adding query filter params (type=image) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:03.207695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8c45bd-c766-419b-a284-4480dcc8a699 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:03.209837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8c45bd-c766-419b-a284-4480dcc8a699 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:03.212157 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11355925-734b-4194-a130-05e968d52e50 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:03.215087 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 509/999] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:19:03 2022] GET /identity/v3/services?type=image => generated 122 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:03.253958 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e83a51-af44-46f6-99cc-4f615fd2afb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:03.254633 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e83a51-af44-46f6-99cc-4f615fd2afb9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:03.254913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e83a51-af44-46f6-99cc-4f615fd2afb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:03.255184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e83a51-af44-46f6-99cc-4f615fd2afb9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:03.256422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e83a51-af44-46f6-99cc-4f615fd2afb9 None admin] RBAC: Adding query filter params (interface=public, service_id=d143a23e6e134796ac3e2f7502d3400a, region_id=RegionOne) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:03.256730 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e83a51-af44-46f6-99cc-4f615fd2afb9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:03.258132 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e83a51-af44-46f6-99cc-4f615fd2afb9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:03.264446 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 491/1000] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Sep 1 16:19:03 2022] GET /identity/v3/endpoints?service_id=d143a23e6e134796ac3e2f7502d3400a&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:19:03.270033 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c45bd-c766-419b-a284-4480dcc8a699 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:03.380002 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c45bd-c766-419b-a284-4480dcc8a699 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:03.400473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:03.414770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50fb3c-50ee-44fe-afc2-7900556e2943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:03.415587 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50fb3c-50ee-44fe-afc2-7900556e2943 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:03.415868 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50fb3c-50ee-44fe-afc2-7900556e2943 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:03.416139 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50fb3c-50ee-44fe-afc2-7900556e2943 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:03.416995 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f50fb3c-50ee-44fe-afc2-7900556e2943 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:03.418377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f50fb3c-50ee-44fe-afc2-7900556e2943 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:03.424626 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 510/1001] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Sep 1 16:19:03 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:19:04.944385 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c02150-1469-4072-9ac9-e457540eaef0 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:04.944930 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c02150-1469-4072-9ac9-e457540eaef0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:04.944930 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c02150-1469-4072-9ac9-e457540eaef0 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:04.947318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 492/1002] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:19:04.951727 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39880b15-a217-4018-9be0-1690a7d031f0 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:04.952145 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39880b15-a217-4018-9be0-1690a7d031f0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:04.952310 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39880b15-a217-4018-9be0-1690a7d031f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:04.971442 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-39880b15-a217-4018-9be0-1690a7d031f0 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:04.972637 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-39880b15-a217-4018-9be0-1690a7d031f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:05.051139 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 511/1003] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:04 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:19:05.124432 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb3ac2-c82e-4b61-b3d1-2ea3639344f3 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:05.124757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb3ac2-c82e-4b61-b3d1-2ea3639344f3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:05.124983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb3ac2-c82e-4b61-b3d1-2ea3639344f3 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:05.126500 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 493/1004] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:19:05.131512 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6660403-ea3b-4958-abfb-7ff24c60b0e0 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:05.131883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6660403-ea3b-4958-abfb-7ff24c60b0e0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:05.132094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6660403-ea3b-4958-abfb-7ff24c60b0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:05.150679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a6660403-ea3b-4958-abfb-7ff24c60b0e0 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:05.151859 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a6660403-ea3b-4958-abfb-7ff24c60b0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:05.220381 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 512/1005] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:05 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:19:05.308918 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d2939-c181-45eb-8330-f060f930dca5 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:05.309253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d2939-c181-45eb-8330-f060f930dca5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:05.309475 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d2939-c181-45eb-8330-f060f930dca5 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:05.310990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 494/1006] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:19:05.315847 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363136f-673f-44ba-a5c1-37b0edc02b8b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:05.316131 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363136f-673f-44ba-a5c1-37b0edc02b8b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:05.316401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363136f-673f-44ba-a5c1-37b0edc02b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:05.323098 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d2939-c181-45eb-8330-f060f930dca5 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:05.335784 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7363136f-673f-44ba-a5c1-37b0edc02b8b None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:05.337150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7363136f-673f-44ba-a5c1-37b0edc02b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:05.368179 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:05.383147 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:05.419164 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 513/1007] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:05 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:19:05.428621 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d2939-c181-45eb-8330-f060f930dca5 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:05.462609 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e2550-e3cf-4048-9810-189fcb4c8c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:05.463237 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e2550-e3cf-4048-9810-189fcb4c8c10 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:05.463521 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e2550-e3cf-4048-9810-189fcb4c8c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:05.463799 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e2550-e3cf-4048-9810-189fcb4c8c10 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:05.464387 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e2550-e3cf-4048-9810-189fcb4c8c10 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:05.465744 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e2550-e3cf-4048-9810-189fcb4c8c10 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:05.474890 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-431e2550-e3cf-4048-9810-189fcb4c8c10 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:19:05.474890 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:05.474890 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:19:05.474890 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:19:05.474890 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:19:05.474890 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:19:05.474890 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:19:05.474890 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:19:05.474890 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:19:05.474890 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:05.474890 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:19:05.474890 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:05.474890 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:05.474890 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:05.474890 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:05.474890 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:05.474890 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:05.475902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:05.475902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:05.475902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:05.475902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:05.475902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:05.475902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:05.475902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:19:05.475902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:19:05.475902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:19:05.475902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:19:05.475902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:05.475902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:05.475902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:05.475902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:05.475902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:05.475902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:05.476799 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:05.476799 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:05.476799 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:05.476799 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:05.476799 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:05.476799 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:05.476799 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:05.476799 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:05.476799 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:05.476799 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:05.476799 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:19:05.476799 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:19:05.476799 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:19:05.476799 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:05.480988 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 495/1008] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Sep 1 16:19:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:19:05.482536 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363136f-673f-44ba-a5c1-37b0edc02b8b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:05.491452 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e2550-e3cf-4048-9810-189fcb4c8c10 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:05.594460 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363136f-673f-44ba-a5c1-37b0edc02b8b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:05.597355 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e2550-e3cf-4048-9810-189fcb4c8c10 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:05.604842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:05.629408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd35ca4-2816-431b-88cf-1d5f18fe8810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:05.630015 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd35ca4-2816-431b-88cf-1d5f18fe8810 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:05.630324 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd35ca4-2816-431b-88cf-1d5f18fe8810 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:05.630599 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd35ca4-2816-431b-88cf-1d5f18fe8810 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:05.631747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd35ca4-2816-431b-88cf-1d5f18fe8810 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:05.632050 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd35ca4-2816-431b-88cf-1d5f18fe8810 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:05.633389 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd35ca4-2816-431b-88cf-1d5f18fe8810 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:05.635509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784262a9-beca-494f-ad40-3ba1bfbff4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:05.636390 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784262a9-beca-494f-ad40-3ba1bfbff4aa None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:05.636818 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784262a9-beca-494f-ad40-3ba1bfbff4aa None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:05.637253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784262a9-beca-494f-ad40-3ba1bfbff4aa None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:05.638044 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784262a9-beca-494f-ad40-3ba1bfbff4aa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:05.638151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 514/1009] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Sep 1 16:19:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:05.639721 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784262a9-beca-494f-ad40-3ba1bfbff4aa None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:05.643622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd35ca4-2816-431b-88cf-1d5f18fe8810 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:05.666240 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 496/1010] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Sep 1 16:19:05 2022] POST /identity/v3/services => generated 242 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:19:05.688339 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:05.693864 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784262a9-beca-494f-ad40-3ba1bfbff4aa None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:05.758408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd35ca4-2816-431b-88cf-1d5f18fe8810 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:05.806916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3ba55-2823-445d-a1e7-f525303ec05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:05.807533 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3ba55-2823-445d-a1e7-f525303ec05e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:05.807810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3ba55-2823-445d-a1e7-f525303ec05e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:05.808083 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3ba55-2823-445d-a1e7-f525303ec05e None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:05.808718 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3ba55-2823-445d-a1e7-f525303ec05e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:05.809017 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3ba55-2823-445d-a1e7-f525303ec05e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:05.810410 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3ba55-2823-445d-a1e7-f525303ec05e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:05.810727 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784262a9-beca-494f-ad40-3ba1bfbff4aa None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:05.816218 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 515/1011] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Sep 1 16:19:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:19:05.823165 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3ba55-2823-445d-a1e7-f525303ec05e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:05.851916 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335ae87-5ec4-444a-a4e0-297e2023f977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:05.852581 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335ae87-5ec4-444a-a4e0-297e2023f977 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:05.852868 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335ae87-5ec4-444a-a4e0-297e2023f977 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:05.853142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335ae87-5ec4-444a-a4e0-297e2023f977 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:05.853732 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4335ae87-5ec4-444a-a4e0-297e2023f977 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:05.855181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4335ae87-5ec4-444a-a4e0-297e2023f977 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:05.864307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-4335ae87-5ec4-444a-a4e0-297e2023f977 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:19:05.864307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:05.864307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:19:05.864307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:19:05.864307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:19:05.864307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:19:05.864307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:19:05.864307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:19:05.864307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:19:05.864307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:05.864307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:19:05.864307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:05.864307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:05.864307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:05.864307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:05.864307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:05.864307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:05.865941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:05.865941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:05.865941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:05.865941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:05.865941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:05.865941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:05.865941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:19:05.865941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:19:05.865941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:19:05.865941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:19:05.865941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:05.865941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:05.865941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:05.865941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:05.865941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:05.865941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:05.867268 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:05.867268 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:05.867268 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:05.867268 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:05.867268 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:05.867268 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:05.867268 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:05.867268 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:05.867268 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:05.867268 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:05.867268 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:19:05.867268 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:19:05.867268 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:19:05.867268 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:05.867268 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 497/1012] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:19:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:19:05.871849 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335ae87-5ec4-444a-a4e0-297e2023f977 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:05.901684 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:05.936956 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:05.941731 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3ba55-2823-445d-a1e7-f525303ec05e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:05.979984 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335ae87-5ec4-444a-a4e0-297e2023f977 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:05.980750 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11f4bb-1909-4972-83bb-b6e443a9e1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:05.981791 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11f4bb-1909-4972-83bb-b6e443a9e1bc None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:05.982349 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11f4bb-1909-4972-83bb-b6e443a9e1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:05.982889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11f4bb-1909-4972-83bb-b6e443a9e1bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:05.983974 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b11f4bb-1909-4972-83bb-b6e443a9e1bc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:05.986264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b11f4bb-1909-4972-83bb-b6e443a9e1bc None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:06.005361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 516/1013] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:19:05 2022] POST /identity/v3/endpoints => generated 325 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:19:06.048356 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4839a7-9cb8-43a5-93a0-c9569802d538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:06.049054 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4839a7-9cb8-43a5-93a0-c9569802d538 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:06.049404 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4839a7-9cb8-43a5-93a0-c9569802d538 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:06.049746 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4839a7-9cb8-43a5-93a0-c9569802d538 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:06.050547 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4839a7-9cb8-43a5-93a0-c9569802d538 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:06.050926 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4839a7-9cb8-43a5-93a0-c9569802d538 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:06.052423 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4839a7-9cb8-43a5-93a0-c9569802d538 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:06.059106 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 498/1014] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Sep 1 16:19:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:19:06.064697 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11f4bb-1909-4972-83bb-b6e443a9e1bc None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:06.178330 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11f4bb-1909-4972-83bb-b6e443a9e1bc None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:06.212679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c18bf-1a08-4691-b4ca-1d2a712753c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:06.213285 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c18bf-1a08-4691-b4ca-1d2a712753c4 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:06.213561 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c18bf-1a08-4691-b4ca-1d2a712753c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:06.213831 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c18bf-1a08-4691-b4ca-1d2a712753c4 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:06.214548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5c18bf-1a08-4691-b4ca-1d2a712753c4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:06.214846 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5c18bf-1a08-4691-b4ca-1d2a712753c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:06.216199 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5c18bf-1a08-4691-b4ca-1d2a712753c4 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:06.221259 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 517/1015] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Sep 1 16:19:06 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:19:06.227223 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4839a7-9cb8-43a5-93a0-c9569802d538 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:06.312624 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:06.329926 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4839a7-9cb8-43a5-93a0-c9569802d538 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:06.364333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e9b0d5-29c0-4292-9793-bad6cb567cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:06.364935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9b0d5-29c0-4292-9793-bad6cb567cf2 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:06.365781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9b0d5-29c0-4292-9793-bad6cb567cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:06.366051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9b0d5-29c0-4292-9793-bad6cb567cf2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:06.366753 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e9b0d5-29c0-4292-9793-bad6cb567cf2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:06.368109 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e9b0d5-29c0-4292-9793-bad6cb567cf2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:06.398223 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 499/1016] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:19:06 2022] POST /identity/v3/endpoints => generated 334 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:19:07.592858 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6adf94-9d1c-4b10-b250-949cd603ff6d None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:07.593380 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6adf94-9d1c-4b10-b250-949cd603ff6d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:07.593380 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6adf94-9d1c-4b10-b250-949cd603ff6d None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:07.594892 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 518/1017] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:19:07.600384 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e45e18-6528-4b7f-939d-92b33104d324 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:07.600784 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e45e18-6528-4b7f-939d-92b33104d324 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:07.600946 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e45e18-6528-4b7f-939d-92b33104d324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:07.619316 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d1e45e18-6528-4b7f-939d-92b33104d324 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:07.620490 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d1e45e18-6528-4b7f-939d-92b33104d324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:07.693888 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 500/1018] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:19:07.956222 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5087f-89b6-47c2-90d7-52d4c0d7c5c7 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:07.956613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5087f-89b6-47c2-90d7-52d4c0d7c5c7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:07.956794 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5087f-89b6-47c2-90d7-52d4c0d7c5c7 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:07.958258 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 519/1019] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:19:07.959573 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e45e18-6528-4b7f-939d-92b33104d324 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:07.963735 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd034f6-efcf-422a-812c-7ee02861e5c0 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:07.964017 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd034f6-efcf-422a-812c-7ee02861e5c0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:07.964291 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd034f6-efcf-422a-812c-7ee02861e5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:07.982751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5bd034f6-efcf-422a-812c-7ee02861e5c0 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:07.984040 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5bd034f6-efcf-422a-812c-7ee02861e5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:08.001532 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:08.028664 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:08.060510 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 520/1020] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:19:08.083371 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e45e18-6528-4b7f-939d-92b33104d324 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:08.119099 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29e110-cbef-4e86-98f5-cb928fdac48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:08.119728 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29e110-cbef-4e86-98f5-cb928fdac48f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:08.120006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29e110-cbef-4e86-98f5-cb928fdac48f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:08.120277 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29e110-cbef-4e86-98f5-cb928fdac48f None admin] PATH_INFO: `/v3/services/image` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:08.120858 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f29e110-cbef-4e86-98f5-cb928fdac48f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:08.122237 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f29e110-cbef-4e86-98f5-cb928fdac48f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:08.131140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-1f29e110-cbef-4e86-98f5-cb928fdac48f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:19:08.131140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:08.131140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:19:08.131140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:19:08.131140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:19:08.131140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:19:08.131140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:19:08.131140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:19:08.131140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:19:08.131140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:08.131140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:19:08.131140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:08.131140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:08.131140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:08.131140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:08.131140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:08.131140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:08.132159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:08.132159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:08.132159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:08.132159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:08.132159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:08.132159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:08.132159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:19:08.132159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:19:08.132159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:19:08.132159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:19:08.132159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:08.132159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:08.132159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:08.132159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:08.132159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:08.132159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:08.135341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:08.135341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:08.135341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:08.135341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:08.135341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:08.135341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:08.135341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:08.135341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:08.135341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:08.135341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:08.135341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:19:08.135341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:19:08.135341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:19:08.135341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:08.135341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 501/1021] 10.222.0.58 () {56 vars in 1105 bytes} [Thu Sep 1 16:19:07 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:19:08.138082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd034f6-efcf-422a-812c-7ee02861e5c0 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:08.240611 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd034f6-efcf-422a-812c-7ee02861e5c0 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:08.274665 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26031647-67de-4f45-91d0-a09a125f84ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:08.275259 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26031647-67de-4f45-91d0-a09a125f84ad None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:08.275534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26031647-67de-4f45-91d0-a09a125f84ad None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:08.275800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26031647-67de-4f45-91d0-a09a125f84ad None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:08.276431 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26031647-67de-4f45-91d0-a09a125f84ad None admin] RBAC: Adding query filter params (name=image) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:08.276728 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26031647-67de-4f45-91d0-a09a125f84ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:08.278081 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26031647-67de-4f45-91d0-a09a125f84ad None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:08.282859 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 521/1022] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:19:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:19:08.289436 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29e110-cbef-4e86-98f5-cb928fdac48f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:08.340743 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26031647-67de-4f45-91d0-a09a125f84ad None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:08.360313 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:08.396820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29e110-cbef-4e86-98f5-cb928fdac48f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:08.425079 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:08.430736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5606244d-b16f-4d60-af1e-a25da0d79b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:08.431338 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606244d-b16f-4d60-af1e-a25da0d79b98 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:08.431619 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606244d-b16f-4d60-af1e-a25da0d79b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:08.431896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606244d-b16f-4d60-af1e-a25da0d79b98 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:08.432529 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5606244d-b16f-4d60-af1e-a25da0d79b98 None admin] RBAC: Adding query filter params (type=image) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:08.432839 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5606244d-b16f-4d60-af1e-a25da0d79b98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:08.434214 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5606244d-b16f-4d60-af1e-a25da0d79b98 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:08.439182 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 502/1023] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:19:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:08.440328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc48fd-ceea-4a02-b906-f372ae2f8409 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:08.440615 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc48fd-ceea-4a02-b906-f372ae2f8409 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:08.440887 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc48fd-ceea-4a02-b906-f372ae2f8409 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:08.446333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 503/1024] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:19:08.447334 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fc48fd-ceea-4a02-b906-f372ae2f8409 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:08.448681 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26031647-67de-4f45-91d0-a09a125f84ad None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:08.482441 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553fdddc-7c07-4a77-b5ca-317a393ac61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:08.483137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fdddc-7c07-4a77-b5ca-317a393ac61a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:08.483473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fdddc-7c07-4a77-b5ca-317a393ac61a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:08.483802 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fdddc-7c07-4a77-b5ca-317a393ac61a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:08.485123 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553fdddc-7c07-4a77-b5ca-317a393ac61a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:08.486790 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553fdddc-7c07-4a77-b5ca-317a393ac61a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:08.488566 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 522/1025] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:19:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:19:08.489820 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005ce3c-ef0f-4428-ae3c-b1355ba4a3e4 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:08.490759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005ce3c-ef0f-4428-ae3c-b1355ba4a3e4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:08.491019 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005ce3c-ef0f-4428-ae3c-b1355ba4a3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:08.508717 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e005ce3c-ef0f-4428-ae3c-b1355ba4a3e4 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:08.509891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e005ce3c-ef0f-4428-ae3c-b1355ba4a3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:08.554113 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fc48fd-ceea-4a02-b906-f372ae2f8409 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:08.575610 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 523/1026] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:19:08.576514 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005ce3c-ef0f-4428-ae3c-b1355ba4a3e4 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:08.586837 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe8a7c-7194-4b4e-be85-9a6bf2613f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:08.587439 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe8a7c-7194-4b4e-be85-9a6bf2613f4d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:08.587719 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe8a7c-7194-4b4e-be85-9a6bf2613f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:08.587990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe8a7c-7194-4b4e-be85-9a6bf2613f4d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:08.588687 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe8a7c-7194-4b4e-be85-9a6bf2613f4d None admin] RBAC: Adding query filter params (interface=public, service_id=77e445bff9104befa535c53aa8259128, region_id=RegionOne) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:08.588994 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe8a7c-7194-4b4e-be85-9a6bf2613f4d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:08.590431 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe8a7c-7194-4b4e-be85-9a6bf2613f4d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:08.595324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 504/1027] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Sep 1 16:19:08 2022] GET /identity/v3/endpoints?service_id=77e445bff9104befa535c53aa8259128&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:08.601156 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe8a7c-7194-4b4e-be85-9a6bf2613f4d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:08.612548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:08.662675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:08.703363 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005ce3c-ef0f-4428-ae3c-b1355ba4a3e4 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:08.705351 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe8a7c-7194-4b4e-be85-9a6bf2613f4d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:08.738790 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47c567-536d-4a62-992a-174c9ed6f84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:08.739383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47c567-536d-4a62-992a-174c9ed6f84f None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:08.739661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47c567-536d-4a62-992a-174c9ed6f84f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:08.739856 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b0c7c-2a85-47aa-9f72-09254dfb6a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:08.740909 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47c567-536d-4a62-992a-174c9ed6f84f None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:08.741015 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b0c7c-2a85-47aa-9f72-09254dfb6a64 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:08.741101 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47c567-536d-4a62-992a-174c9ed6f84f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:08.741379 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b0c7c-2a85-47aa-9f72-09254dfb6a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:08.741567 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b0c7c-2a85-47aa-9f72-09254dfb6a64 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:08.742147 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43b0c7c-2a85-47aa-9f72-09254dfb6a64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:08.742609 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47c567-536d-4a62-992a-174c9ed6f84f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:08.743551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43b0c7c-2a85-47aa-9f72-09254dfb6a64 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:08.750289 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 505/1028] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Sep 1 16:19:08 2022] GET /identity/v3/services => generated 1957 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:19:08.782236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 524/1029] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Sep 1 16:19:08 2022] POST /identity/v3/projects => generated 311 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:19:08.845531 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b0c7c-2a85-47aa-9f72-09254dfb6a64 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:08.960269 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b0c7c-2a85-47aa-9f72-09254dfb6a64 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:08.985725 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:08.997330 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff06070-f27f-4c69-b943-f47c53602ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:08.997891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff06070-f27f-4c69-b943-f47c53602ab0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:08.998172 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff06070-f27f-4c69-b943-f47c53602ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:08.998528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff06070-f27f-4c69-b943-f47c53602ab0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:09.009946 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff06070-f27f-4c69-b943-f47c53602ab0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:19:09.014756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-aff06070-f27f-4c69-b943-f47c53602ab0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:19:09.014756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:09.014756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:09.014756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:09.014756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:09.014756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:09.014756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:09.014756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:09.014756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:09.014756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:09.014756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:09.014756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:09.014756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:19:09.014756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:19:09.014756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:19:09.014756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:19:09.015774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:19:09.015774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:19:09.015774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:19:09.015774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:19:09.015774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:09.015774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:09.015774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:09.015774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:09.015774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:09.015774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:09.015774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:09.015774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:09.015774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:09.015774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:09.015774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:09.015774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:09.016673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:09.016673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:09.016673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:09.016673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:09.016673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:19:09.016673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:19:09.016673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:19:09.016673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:19:09.016673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:19:09.016673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:19:09.016673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:19:09.016673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:09.016673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 506/1030] 10.222.0.58 () {56 vars in 1120 bytes} [Thu Sep 1 16:19:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:19:09.022502 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47c567-536d-4a62-992a-174c9ed6f84f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:09.096948 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:09.134501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47c567-536d-4a62-992a-174c9ed6f84f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:09.170643 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb70c7-5719-4865-b2e0-a9647e19c520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:09.171344 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb70c7-5719-4865-b2e0-a9647e19c520 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:09.171630 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb70c7-5719-4865-b2e0-a9647e19c520 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:09.171908 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb70c7-5719-4865-b2e0-a9647e19c520 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:09.172589 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb70c7-5719-4865-b2e0-a9647e19c520 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:09.172897 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb70c7-5719-4865-b2e0-a9647e19c520 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:09.174293 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb70c7-5719-4865-b2e0-a9647e19c520 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:09.183010 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 525/1031] 10.222.0.58 () {56 vars in 1115 bytes} [Thu Sep 1 16:19:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:09.188805 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff06070-f27f-4c69-b943-f47c53602ab0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:09.298311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff06070-f27f-4c69-b943-f47c53602ab0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:09.331587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bfc3cc-ee60-4904-991c-09c28a049940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:09.332212 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfc3cc-ee60-4904-991c-09c28a049940 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:09.332491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfc3cc-ee60-4904-991c-09c28a049940 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:09.332759 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfc3cc-ee60-4904-991c-09c28a049940 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:09.343654 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bfc3cc-ee60-4904-991c-09c28a049940 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:09.345088 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bfc3cc-ee60-4904-991c-09c28a049940 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:09.359040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-32bfc3cc-ee60-4904-991c-09c28a049940 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:19:09.359040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:09.359040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:09.359040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:09.359040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:09.359040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:09.359040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:09.359040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:09.359040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:09.359040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:09.359040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:09.359040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:09.359040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:19:09.359040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:19:09.359040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:19:09.359040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:19:09.360027 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:09.360027 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:09.360027 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:09.360027 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:09.360027 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:09.360027 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:09.360027 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:09.360027 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:09.360027 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:09.360027 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:09.360027 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:09.360027 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:09.360027 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:09.360027 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:09.360027 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:09.360027 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:09.360027 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:19:09.360951 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:19:09.360951 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:19:09.360951 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:09.360951 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 507/1032] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:19:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:19:09.367361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb70c7-5719-4865-b2e0-a9647e19c520 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:09.476866 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb70c7-5719-4865-b2e0-a9647e19c520 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:09.502910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:09.512564 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f963e9-8ff6-468d-a468-3eb187c60c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:09.513164 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f963e9-8ff6-468d-a468-3eb187c60c83 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:09.513442 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f963e9-8ff6-468d-a468-3eb187c60c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:09.513710 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f963e9-8ff6-468d-a468-3eb187c60c83 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:09.514378 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f963e9-8ff6-468d-a468-3eb187c60c83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:09.514681 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f963e9-8ff6-468d-a468-3eb187c60c83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:09.516037 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f963e9-8ff6-468d-a468-3eb187c60c83 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:09.528981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 526/1033] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:09.534433 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bfc3cc-ee60-4904-991c-09c28a049940 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:09.574428 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:09.640175 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bfc3cc-ee60-4904-991c-09c28a049940 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:09.673758 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaebdf2-30c5-4f70-b289-4a20b151df36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:09.674416 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaebdf2-30c5-4f70-b289-4a20b151df36 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:09.674698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaebdf2-30c5-4f70-b289-4a20b151df36 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:09.674970 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaebdf2-30c5-4f70-b289-4a20b151df36 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:09.682349 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaebdf2-30c5-4f70-b289-4a20b151df36 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:09.683838 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaebdf2-30c5-4f70-b289-4a20b151df36 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:09.694898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-dcaebdf2-30c5-4f70-b289-4a20b151df36 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:19:09.694898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:09.694898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:09.694898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:09.694898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:09.694898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:09.694898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:09.694898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:09.694898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:09.694898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:09.694898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:09.694898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:09.694898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:19:09.694898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:19:09.694898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:19:09.694898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:19:09.695858 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:09.695858 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:09.695858 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:19:09.695858 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:19:09.695858 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:19:09.695858 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:19:09.695858 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:09.695858 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:09.695858 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:09.695858 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:09.695858 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:09.695858 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:09.695858 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:09.695858 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:09.695858 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:09.695858 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:09.695858 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:09.695858 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:09.696824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:09.696824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:09.696824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:19:09.696824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:19:09.696824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:19:09.696824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:19:09.696824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:19:09.696824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:19:09.696824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:19:09.696824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:09.696824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 508/1034] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Sep 1 16:19:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:19:09.703553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f963e9-8ff6-468d-a468-3eb187c60c83 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:09.804523 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f963e9-8ff6-468d-a468-3eb187c60c83 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:09.839894 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c501c6-aa83-4775-b83e-f8e2c691b51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:09.840756 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c501c6-aa83-4775-b83e-f8e2c691b51c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:09.841281 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c501c6-aa83-4775-b83e-f8e2c691b51c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:09.841718 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c501c6-aa83-4775-b83e-f8e2c691b51c None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:09.842670 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c501c6-aa83-4775-b83e-f8e2c691b51c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:09.843126 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c501c6-aa83-4775-b83e-f8e2c691b51c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:09.844714 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c501c6-aa83-4775-b83e-f8e2c691b51c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:09.863454 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 527/1035] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Sep 1 16:19:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:09.869243 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaebdf2-30c5-4f70-b289-4a20b151df36 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:09.967020 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:09.974077 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaebdf2-30c5-4f70-b289-4a20b151df36 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:10.007785 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcae86-8516-4d90-9fac-c17375a668ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:10.008376 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcae86-8516-4d90-9fac-c17375a668ba None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:10.008652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcae86-8516-4d90-9fac-c17375a668ba None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:10.008921 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcae86-8516-4d90-9fac-c17375a668ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:10.081974 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bcae86-8516-4d90-9fac-c17375a668ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:10.083483 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bcae86-8516-4d90-9fac-c17375a668ba None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:10.176238 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:10.189139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 509/1036] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:19:09 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:19:10.190753 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c501c6-aa83-4775-b83e-f8e2c691b51c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:10.239610 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:10.294041 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c501c6-aa83-4775-b83e-f8e2c691b51c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:10.328442 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b9d2c-2783-44ed-91bd-260a7cb0abef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:10.329066 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b9d2c-2783-44ed-91bd-260a7cb0abef None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:10.329338 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b9d2c-2783-44ed-91bd-260a7cb0abef None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:10.329603 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b9d2c-2783-44ed-91bd-260a7cb0abef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:10.339679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b9d2c-2783-44ed-91bd-260a7cb0abef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:10.341182 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b9d2c-2783-44ed-91bd-260a7cb0abef None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:10.356150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-ba0b9d2c-2783-44ed-91bd-260a7cb0abef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:19:10.356150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:10.356150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:10.356150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:10.356150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:10.356150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:10.356150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:10.356150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:10.356150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:10.356150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:10.356150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:10.356150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:10.356150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:19:10.356150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:19:10.356150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:19:10.356150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:19:10.357166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:10.357166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:10.357166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:10.357166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:10.357166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:10.357166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:10.357166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:10.357166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:10.357166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:10.357166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:10.357166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:10.357166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:10.357166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:10.357166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:10.357166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:10.357166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:10.357166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:19:10.358109 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:19:10.358109 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:19:10.358109 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:19:10.361863 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 528/1037] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:19:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:19:10.363288 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcae86-8516-4d90-9fac-c17375a668ba None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:10.468670 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcae86-8516-4d90-9fac-c17375a668ba None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:10.503021 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b208f-816b-46bb-8a19-53f2ecd1d6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:10.503615 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b208f-816b-46bb-8a19-53f2ecd1d6d4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:10.503893 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b208f-816b-46bb-8a19-53f2ecd1d6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:10.504164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b208f-816b-46bb-8a19-53f2ecd1d6d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:10.504805 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b208f-816b-46bb-8a19-53f2ecd1d6d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:10.505106 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b208f-816b-46bb-8a19-53f2ecd1d6d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:10.507170 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b208f-816b-46bb-8a19-53f2ecd1d6d4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:10.522889 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 510/1038] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:19:10.524622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b9d2c-2783-44ed-91bd-260a7cb0abef None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:10.623214 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:10.628146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b9d2c-2783-44ed-91bd-260a7cb0abef None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:10.664247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870899fa-636c-40f8-85da-bcd6c54aff3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:10.664876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870899fa-636c-40f8-85da-bcd6c54aff3e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:10.665159 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870899fa-636c-40f8-85da-bcd6c54aff3e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:10.665433 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870899fa-636c-40f8-85da-bcd6c54aff3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:10.669883 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941654b-8d86-46f7-b34e-ba29c65b02cb None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:10.670348 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941654b-8d86-46f7-b34e-ba29c65b02cb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:10.670536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941654b-8d86-46f7-b34e-ba29c65b02cb None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:10.671938 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 511/1039] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:19:10.673121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870899fa-636c-40f8-85da-bcd6c54aff3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:10.674595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870899fa-636c-40f8-85da-bcd6c54aff3e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:10.676879 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d06b0-0838-4ed6-8cee-43c78a352295 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:10.677220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d06b0-0838-4ed6-8cee-43c78a352295 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:10.678023 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d06b0-0838-4ed6-8cee-43c78a352295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:10.687195 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-870899fa-636c-40f8-85da-bcd6c54aff3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:19:10.687195 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:10.687195 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:10.687195 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:10.687195 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:10.687195 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:10.687195 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:10.687195 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:10.687195 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:10.687195 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:10.687195 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:10.687195 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:10.687195 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:19:10.687195 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:19:10.687195 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:19:10.688210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:19:10.688210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:10.688210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:10.688210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:10.688210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:10.688210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:10.688210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:10.688210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:10.688210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:10.688210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:10.688210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:10.688210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:10.688210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:10.688210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:10.688210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:10.688210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:10.688210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:10.689136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:19:10.689136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:19:10.689136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:19:10.689136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:19:10.689136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:19:10.689136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:19:10.689136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:19:10.689136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:19:10.689136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 529/1040] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:19:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:19:10.694704 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870899fa-636c-40f8-85da-bcd6c54aff3e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:10.701323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-218d06b0-0838-4ed6-8cee-43c78a352295 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:10.702553 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-218d06b0-0838-4ed6-8cee-43c78a352295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:10.759087 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:10.776851 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 512/1041] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:19:10.783057 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671e6a0-93df-4b77-bd28-1d81c30d1d03 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:10.783351 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671e6a0-93df-4b77-bd28-1d81c30d1d03 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:10.783634 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671e6a0-93df-4b77-bd28-1d81c30d1d03 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:10.785058 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 513/1042] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:19:10.790185 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78563458-227a-43bf-85d0-847cb3df54c7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:10.790499 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78563458-227a-43bf-85d0-847cb3df54c7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:10.790778 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78563458-227a-43bf-85d0-847cb3df54c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:10.809673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-78563458-227a-43bf-85d0-847cb3df54c7 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:10.810892 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-78563458-227a-43bf-85d0-847cb3df54c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:10.812189 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870899fa-636c-40f8-85da-bcd6c54aff3e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:10.850077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4aa36-189b-43db-8382-86fd76cf1c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:10.850712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4aa36-189b-43db-8382-86fd76cf1c7d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:10.850993 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4aa36-189b-43db-8382-86fd76cf1c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:10.851273 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4aa36-189b-43db-8382-86fd76cf1c7d None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:10.851966 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c4aa36-189b-43db-8382-86fd76cf1c7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:10.852272 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c4aa36-189b-43db-8382-86fd76cf1c7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:10.853694 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c4aa36-189b-43db-8382-86fd76cf1c7d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:10.867239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 530/1043] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:19:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:19:10.872963 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4aa36-189b-43db-8382-86fd76cf1c7d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:10.881679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 514/1044] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:19:10.892870 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:10.976938 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4aa36-189b-43db-8382-86fd76cf1c7d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:11.011423 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d9cf9-f459-4069-8c47-6ddafd9c10ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:11.011960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d9cf9-f459-4069-8c47-6ddafd9c10ca None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:11.012243 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d9cf9-f459-4069-8c47-6ddafd9c10ca None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:11.012518 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d9cf9-f459-4069-8c47-6ddafd9c10ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:11.013247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d9cf9-f459-4069-8c47-6ddafd9c10ca None admin] RBAC: Adding query filter params (role.id=1ff98ccdce644654b43e48dc28fce36b, scope.project.id=479530ced580426b85d1fc07af43417f, user.id=1f0c99fdf8bd4331ad7e641ee38554c1) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:11.013553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d9cf9-f459-4069-8c47-6ddafd9c10ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:11.015069 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d9cf9-f459-4069-8c47-6ddafd9c10ca None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:11.022389 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 531/1045] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:19:10 2022] GET /identity/v3/role_assignments?user.id=1f0c99fdf8bd4331ad7e641ee38554c1&scope.project.id=479530ced580426b85d1fc07af43417f&role.id=1ff98ccdce644654b43e48dc28fce36b => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:19:11.056000 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78563458-227a-43bf-85d0-847cb3df54c7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:11.152779 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d9cf9-f459-4069-8c47-6ddafd9c10ca None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:11.175447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78563458-227a-43bf-85d0-847cb3df54c7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:11.193884 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:11.213493 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c912f-204d-4813-ae1e-6c1a88beef37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:11.214137 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c912f-204d-4813-ae1e-6c1a88beef37 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:11.214456 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c912f-204d-4813-ae1e-6c1a88beef37 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:11.214735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c912f-204d-4813-ae1e-6c1a88beef37 None admin] PATH_INFO: `/v3/services/image` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:11.215342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c912f-204d-4813-ae1e-6c1a88beef37 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:11.216706 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c912f-204d-4813-ae1e-6c1a88beef37 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:11.227074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-221c912f-204d-4813-ae1e-6c1a88beef37 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:19:11.227074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:11.227074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:19:11.227074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:19:11.227074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:19:11.227074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:19:11.227074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:19:11.227074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:19:11.227074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:19:11.227074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:11.227074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:19:11.227074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:11.227074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:11.227074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:11.227074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:11.227074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:11.227074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:11.228187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:11.228187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:11.228187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:11.228187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:11.228187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:11.228187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:11.228187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:19:11.228187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:19:11.228187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:19:11.228187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:19:11.228187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:11.228187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:11.228187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:11.228187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:11.228187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:11.228187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:11.229092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:11.229092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:11.229092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:11.229092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:11.229092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:11.229092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:11.229092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:11.229092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:11.229092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:11.229092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:11.229092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:19:11.229092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:19:11.229092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:19:11.229092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:11.234283 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 515/1046] 10.222.0.58 () {56 vars in 1105 bytes} [Thu Sep 1 16:19:11 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:19:11.236145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c912f-204d-4813-ae1e-6c1a88beef37 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:11.247985 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:11.270072 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d9cf9-f459-4069-8c47-6ddafd9c10ca None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:11.304951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da924a-e772-472a-8986-968d1c2a6eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:11.305581 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da924a-e772-472a-8986-968d1c2a6eaf None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:11.305863 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da924a-e772-472a-8986-968d1c2a6eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:11.306139 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da924a-e772-472a-8986-968d1c2a6eaf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:11.307305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10da924a-e772-472a-8986-968d1c2a6eaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:11.308732 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10da924a-e772-472a-8986-968d1c2a6eaf None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:11.310415 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 532/1047] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:19:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:19:11.315994 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da924a-e772-472a-8986-968d1c2a6eaf None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:11.351681 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c912f-204d-4813-ae1e-6c1a88beef37 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:11.393194 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60752381-c5f1-41ac-bd49-21ba66316d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:11.393999 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60752381-c5f1-41ac-bd49-21ba66316d71 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:11.394313 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60752381-c5f1-41ac-bd49-21ba66316d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:11.394585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60752381-c5f1-41ac-bd49-21ba66316d71 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:11.395227 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60752381-c5f1-41ac-bd49-21ba66316d71 None admin] RBAC: Adding query filter params (name=image) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:11.395518 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60752381-c5f1-41ac-bd49-21ba66316d71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:11.396918 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60752381-c5f1-41ac-bd49-21ba66316d71 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:11.402815 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 516/1048] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:19:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:11.408731 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60752381-c5f1-41ac-bd49-21ba66316d71 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:11.441769 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da924a-e772-472a-8986-968d1c2a6eaf None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:11.457183 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:11.480190 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49e981-27db-4eef-800d-8e3038dcfb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:11.481185 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49e981-27db-4eef-800d-8e3038dcfb49 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:11.481511 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49e981-27db-4eef-800d-8e3038dcfb49 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:11.481811 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49e981-27db-4eef-800d-8e3038dcfb49 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:11.482649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff49e981-27db-4eef-800d-8e3038dcfb49 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:11.484493 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff49e981-27db-4eef-800d-8e3038dcfb49 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:11.514232 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 533/1049] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Sep 1 16:19:11 2022] POST /identity/v3/users => generated 285 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:19:11.516843 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60752381-c5f1-41ac-bd49-21ba66316d71 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:11.558401 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c9d811-d3d2-40b3-baee-ede5bcdb6e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:11.558942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9d811-d3d2-40b3-baee-ede5bcdb6e67 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:11.559219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9d811-d3d2-40b3-baee-ede5bcdb6e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:11.559489 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9d811-d3d2-40b3-baee-ede5bcdb6e67 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:11.560142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c9d811-d3d2-40b3-baee-ede5bcdb6e67 None admin] RBAC: Adding query filter params (type=image) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:11.560448 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c9d811-d3d2-40b3-baee-ede5bcdb6e67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:11.561856 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c9d811-d3d2-40b3-baee-ede5bcdb6e67 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:11.572515 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 517/1050] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:19:11 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:11.574400 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49e981-27db-4eef-800d-8e3038dcfb49 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:11.585244 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:11.689579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49e981-27db-4eef-800d-8e3038dcfb49 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:11.724595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b41175-f343-4d81-b9ac-fba24ee59af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:11.725211 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b41175-f343-4d81-b9ac-fba24ee59af5 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:11.725555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b41175-f343-4d81-b9ac-fba24ee59af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:11.725838 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b41175-f343-4d81-b9ac-fba24ee59af5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:11.726597 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b41175-f343-4d81-b9ac-fba24ee59af5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:11.727987 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b41175-f343-4d81-b9ac-fba24ee59af5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:11.753508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 534/1051] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:19:11 2022] POST /identity/v3/endpoints => generated 327 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:19:12.935324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d5677-cbb5-46f2-9088-b20e3199803b None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:12.935871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d5677-cbb5-46f2-9088-b20e3199803b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:12.935871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d5677-cbb5-46f2-9088-b20e3199803b None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:12.937219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 518/1052] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:19:12.942606 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7f778-373d-4d82-9ba7-7574e490db25 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:12.942988 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7f778-373d-4d82-9ba7-7574e490db25 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:12.943218 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7f778-373d-4d82-9ba7-7574e490db25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:12.962137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2be7f778-373d-4d82-9ba7-7574e490db25 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:12.963337 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2be7f778-373d-4d82-9ba7-7574e490db25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:13.019342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:13.038248 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 535/1053] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:19:13.305254 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d5677-cbb5-46f2-9088-b20e3199803b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:13.409317 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d5677-cbb5-46f2-9088-b20e3199803b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:13.420055 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:13.445500 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d35f6-843e-411b-84de-14bb1abeaf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:13.446128 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d35f6-843e-411b-84de-14bb1abeaf18 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:13.446436 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d35f6-843e-411b-84de-14bb1abeaf18 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:13.446774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d35f6-843e-411b-84de-14bb1abeaf18 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:13.457370 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d35f6-843e-411b-84de-14bb1abeaf18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:19:13.462711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-2e0d35f6-843e-411b-84de-14bb1abeaf18 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:19:13.462711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:13.462711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:13.462711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:13.462711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:13.462711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:13.462711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:13.462711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:13.462711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:13.462711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:13.462711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:13.462711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:13.462711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:19:13.462711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:19:13.462711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:19:13.462711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:19:13.463672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:19:13.463672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:19:13.463672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:19:13.463672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:19:13.463672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:13.463672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:13.463672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:13.463672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:13.463672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:13.463672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:13.463672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:13.463672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:13.463672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:13.463672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:13.463672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:13.463672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:13.464632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:13.464632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:13.464632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:13.464632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:13.464632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:19:13.464632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:19:13.464632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:19:13.464632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:19:13.464632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:19:13.464632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:19:13.464632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:19:13.464632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:13.466553 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 519/1054] 10.222.0.58 () {56 vars in 1120 bytes} [Thu Sep 1 16:19:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:19:13.469752 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be7f778-373d-4d82-9ba7-7574e490db25 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:13.495151 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86a5b8-2600-4f7f-935a-b4fdd384efac None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:13.495441 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86a5b8-2600-4f7f-935a-b4fdd384efac None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:13.495717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86a5b8-2600-4f7f-935a-b4fdd384efac None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:13.497187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 520/1055] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:19:13.501790 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312b556-b2fe-4bc6-9841-ad91f0deb311 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:13.502079 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312b556-b2fe-4bc6-9841-ad91f0deb311 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:13.502384 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312b556-b2fe-4bc6-9841-ad91f0deb311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:13.520104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9312b556-b2fe-4bc6-9841-ad91f0deb311 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:13.521812 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9312b556-b2fe-4bc6-9841-ad91f0deb311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:13.582455 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be7f778-373d-4d82-9ba7-7574e490db25 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:13.600481 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 521/1056] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:19:13.618991 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4b144-e583-4465-9310-bebb99f045e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:13.619835 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4b144-e583-4465-9310-bebb99f045e0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:13.620293 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4b144-e583-4465-9310-bebb99f045e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:13.620745 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4b144-e583-4465-9310-bebb99f045e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:13.621649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a4b144-e583-4465-9310-bebb99f045e0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:13.622132 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a4b144-e583-4465-9310-bebb99f045e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:13.623700 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a4b144-e583-4465-9310-bebb99f045e0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:13.632874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 536/1057] 10.222.0.58 () {56 vars in 1115 bytes} [Thu Sep 1 16:19:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:13.638185 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312b556-b2fe-4bc6-9841-ad91f0deb311 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:13.729526 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a49f1-9e45-4ca6-ba22-80f49f55c30f None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:13.729851 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a49f1-9e45-4ca6-ba22-80f49f55c30f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:13.730084 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a49f1-9e45-4ca6-ba22-80f49f55c30f None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:13.734239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 537/1058] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:19:13.734374 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:13.736896 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f9964-f0f3-464a-858b-a1b8ef1a666f None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:13.737274 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f9964-f0f3-464a-858b-a1b8ef1a666f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:13.737647 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f9964-f0f3-464a-858b-a1b8ef1a666f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:13.737881 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312b556-b2fe-4bc6-9841-ad91f0deb311 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:13.756907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-627f9964-f0f3-464a-858b-a1b8ef1a666f None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:13.758175 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-627f9964-f0f3-464a-858b-a1b8ef1a666f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:13.781422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca24614-795c-4d8a-bd89-ef58549fd206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:13.782604 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca24614-795c-4d8a-bd89-ef58549fd206 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:13.782956 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca24614-795c-4d8a-bd89-ef58549fd206 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:13.783574 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca24614-795c-4d8a-bd89-ef58549fd206 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:13.797749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca24614-795c-4d8a-bd89-ef58549fd206 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:13.799667 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca24614-795c-4d8a-bd89-ef58549fd206 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:13.806729 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:13.821051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-cca24614-795c-4d8a-bd89-ef58549fd206 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:19:13.821051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:13.821051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:13.821051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:13.821051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:13.821051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:13.821051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:13.821051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:13.821051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:13.821051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:13.821051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:13.821051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:13.821051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:19:13.821051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:19:13.821051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:19:13.821051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:19:13.822038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:13.822038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:13.822038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:13.822038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:13.822038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:13.822038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:13.822038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:13.822038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:13.822038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:13.822038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:13.822038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:13.822038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:13.822038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:13.822038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:13.822038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:13.822038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:13.822038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:19:13.823002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:19:13.823002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:19:13.823002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:13.827657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 522/1059] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:19:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:19:13.828415 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca24614-795c-4d8a-bd89-ef58549fd206 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:13.829179 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 538/1060] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:19:13.873385 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f9964-f0f3-464a-858b-a1b8ef1a666f None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:13.935584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca24614-795c-4d8a-bd89-ef58549fd206 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:13.973000 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff787c-9b45-4077-9b8f-cc30d9876a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:13.973605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff787c-9b45-4077-9b8f-cc30d9876a16 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:13.973889 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff787c-9b45-4077-9b8f-cc30d9876a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:13.974220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff787c-9b45-4077-9b8f-cc30d9876a16 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:13.974859 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff787c-9b45-4077-9b8f-cc30d9876a16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:13.975166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff787c-9b45-4077-9b8f-cc30d9876a16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:13.976534 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff787c-9b45-4077-9b8f-cc30d9876a16 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:13.978255 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f9964-f0f3-464a-858b-a1b8ef1a666f None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:13.992963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 523/1061] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:19:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:19:13.994006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff787c-9b45-4077-9b8f-cc30d9876a16 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:14.006259 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:14.018685 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de8220-afdf-44fd-8bbe-b2d9a01d06f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:14.019328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de8220-afdf-44fd-8bbe-b2d9a01d06f0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:14.019621 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de8220-afdf-44fd-8bbe-b2d9a01d06f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:14.019883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de8220-afdf-44fd-8bbe-b2d9a01d06f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:14.030821 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de8220-afdf-44fd-8bbe-b2d9a01d06f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:19:14.035545 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-83de8220-afdf-44fd-8bbe-b2d9a01d06f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:19:14.035545 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:14.035545 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:14.035545 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:14.035545 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:14.035545 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:14.035545 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:14.035545 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:14.035545 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:14.035545 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:14.035545 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:14.035545 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:14.035545 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:19:14.035545 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:19:14.035545 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:19:14.035545 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:19:14.036755 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:19:14.036755 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:19:14.036755 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:19:14.036755 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:19:14.036755 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:14.036755 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:14.036755 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:14.036755 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:14.036755 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:14.036755 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:14.036755 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:14.036755 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:14.036755 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:14.036755 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:14.036755 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:14.036755 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:14.037841 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:14.037841 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:14.037841 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:14.037841 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:14.037841 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:19:14.037841 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:19:14.037841 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:19:14.037841 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:19:14.037841 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:19:14.037841 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:19:14.037841 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:19:14.037841 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:19:14.037841 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 539/1062] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:19:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:19:14.043171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de8220-afdf-44fd-8bbe-b2d9a01d06f0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:14.088340 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:14.114248 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff787c-9b45-4077-9b8f-cc30d9876a16 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:14.147489 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de8220-afdf-44fd-8bbe-b2d9a01d06f0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:14.149688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527344e-f824-4a91-b123-e7520fb795e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:14.150640 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527344e-f824-4a91-b123-e7520fb795e4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:14.151055 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527344e-f824-4a91-b123-e7520fb795e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:14.151460 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527344e-f824-4a91-b123-e7520fb795e4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:14.160653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5527344e-f824-4a91-b123-e7520fb795e4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:14.162769 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5527344e-f824-4a91-b123-e7520fb795e4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:14.174539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-5527344e-f824-4a91-b123-e7520fb795e4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:19:14.174539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:14.174539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:14.174539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:14.174539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:14.174539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:14.174539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:14.174539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:14.174539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:14.174539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:14.174539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:14.174539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:14.174539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:19:14.174539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:19:14.174539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:19:14.174539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:19:14.175569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:14.175569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:14.175569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:19:14.175569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:19:14.175569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:19:14.175569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:19:14.175569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:14.175569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:14.175569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:14.175569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:14.175569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:14.175569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:14.175569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:14.175569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:14.175569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:14.175569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:14.175569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:14.175569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:14.176561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:14.176561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:14.176561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:19:14.176561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:19:14.176561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:19:14.176561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:19:14.176561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:19:14.176561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:19:14.176561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:19:14.176561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:14.176561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 524/1063] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Sep 1 16:19:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:19:14.176561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527344e-f824-4a91-b123-e7520fb795e4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:14.187253 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec1626-688a-4113-bae4-9a126afa2f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:14.187851 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec1626-688a-4113-bae4-9a126afa2f33 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:14.188128 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec1626-688a-4113-bae4-9a126afa2f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:14.188396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec1626-688a-4113-bae4-9a126afa2f33 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:14.189055 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccec1626-688a-4113-bae4-9a126afa2f33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:14.189352 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccec1626-688a-4113-bae4-9a126afa2f33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:14.190728 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccec1626-688a-4113-bae4-9a126afa2f33 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:14.203298 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 540/1064] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:19:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:14.203298 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec1626-688a-4113-bae4-9a126afa2f33 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:14.250443 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:14.281345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527344e-f824-4a91-b123-e7520fb795e4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:14.297370 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec1626-688a-4113-bae4-9a126afa2f33 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:14.298902 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:14.316735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea00600-bf20-49e0-b12a-d2691594c801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:14.317337 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea00600-bf20-49e0-b12a-d2691594c801 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:14.317620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea00600-bf20-49e0-b12a-d2691594c801 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:14.317916 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea00600-bf20-49e0-b12a-d2691594c801 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:14.330905 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869376c-bf0b-44f7-a5f2-1b5e24811836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:14.331508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869376c-bf0b-44f7-a5f2-1b5e24811836 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:14.331779 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869376c-bf0b-44f7-a5f2-1b5e24811836 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:14.332056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869376c-bf0b-44f7-a5f2-1b5e24811836 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:14.332789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869376c-bf0b-44f7-a5f2-1b5e24811836 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:14.333092 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869376c-bf0b-44f7-a5f2-1b5e24811836 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:14.335037 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869376c-bf0b-44f7-a5f2-1b5e24811836 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:14.352334 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 541/1065] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Sep 1 16:19:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:14.353202 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869376c-bf0b-44f7-a5f2-1b5e24811836 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:14.388385 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea00600-bf20-49e0-b12a-d2691594c801 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:14.389814 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea00600-bf20-49e0-b12a-d2691594c801 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:14.448368 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869376c-bf0b-44f7-a5f2-1b5e24811836 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:14.458165 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:14.483534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4aa056-017c-4eba-bbc2-da7462c543cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:14.484157 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4aa056-017c-4eba-bbc2-da7462c543cf None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:14.484435 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4aa056-017c-4eba-bbc2-da7462c543cf None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:14.484711 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4aa056-017c-4eba-bbc2-da7462c543cf None admin] PATH_INFO: `/v3/users/c73961187bf449c18c2cc14f438650b9` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:14.491323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 525/1066] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:19:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:19:14.492152 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea00600-bf20-49e0-b12a-d2691594c801 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:14.498782 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4aa056-017c-4eba-bbc2-da7462c543cf None admin] RBAC: Authorizing `identity:get_user(user_id=c73961187bf449c18c2cc14f438650b9)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:14.500223 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4aa056-017c-4eba-bbc2-da7462c543cf None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:14.506230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 542/1067] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:19:14 2022] GET /identity/v3/users/c73961187bf449c18c2cc14f438650b9 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:14.506230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4aa056-017c-4eba-bbc2-da7462c543cf None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:14.521564 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:14.594219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea00600-bf20-49e0-b12a-d2691594c801 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:14.602207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4aa056-017c-4eba-bbc2-da7462c543cf None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:14.627609 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ebbc6-e3c6-4bda-9a44-e8c972a9ebaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:14.628222 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ebbc6-e3c6-4bda-9a44-e8c972a9ebaf None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:14.628499 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ebbc6-e3c6-4bda-9a44-e8c972a9ebaf None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:14.628765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ebbc6-e3c6-4bda-9a44-e8c972a9ebaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:14.635378 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9ddc7-ef20-4fba-8f7f-83bfb49d894b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:14.635982 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9ddc7-ef20-4fba-8f7f-83bfb49d894b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:14.636250 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9ddc7-ef20-4fba-8f7f-83bfb49d894b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:14.636520 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9ddc7-ef20-4fba-8f7f-83bfb49d894b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:14.639257 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ebbc6-e3c6-4bda-9a44-e8c972a9ebaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:14.640669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ebbc6-e3c6-4bda-9a44-e8c972a9ebaf None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:14.647269 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9ddc7-ef20-4fba-8f7f-83bfb49d894b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:14.649073 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9ddc7-ef20-4fba-8f7f-83bfb49d894b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:14.655593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-9c1ebbc6-e3c6-4bda-9a44-e8c972a9ebaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:19:14.655593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:14.655593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:14.655593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:14.655593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:14.655593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:14.655593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:14.655593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:14.655593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:14.655593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:14.655593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:14.655593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:14.655593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:19:14.655593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:19:14.655593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:19:14.655593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:19:14.656760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:14.656760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:14.656760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:14.656760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:14.656760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:14.656760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:14.656760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:14.656760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:14.656760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:14.656760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:14.656760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:14.656760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:14.656760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:14.656760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:14.656760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:14.656760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:14.656760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:19:14.657908 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:19:14.657908 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:19:14.657908 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:14.657908 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 526/1068] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:19:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:19:14.657908 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ebbc6-e3c6-4bda-9a44-e8c972a9ebaf None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:14.670121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-08c9ddc7-ef20-4fba-8f7f-83bfb49d894b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:19:14.670121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:14.670121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:14.670121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:14.670121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:14.670121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:14.670121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:14.670121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:14.670121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:14.670121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:14.670121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:14.670121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:14.670121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:19:14.670121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:19:14.670121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:19:14.670121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:19:14.671299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:14.671299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:14.671299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:14.671299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:14.671299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:14.671299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:14.671299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:14.671299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:14.671299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:14.671299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:14.671299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:14.671299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:14.671299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:14.671299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:14.671299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:14.671299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:14.671299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:19:14.672648 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:19:14.672648 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:19:14.672648 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:19:14.672648 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 543/1069] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:19:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:19:14.672648 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9ddc7-ef20-4fba-8f7f-83bfb49d894b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:14.684752 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:14.742645 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:14.759477 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ebbc6-e3c6-4bda-9a44-e8c972a9ebaf None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:14.773133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9ddc7-ef20-4fba-8f7f-83bfb49d894b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:14.794215 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd4464-fca2-43ad-9c3d-1b3aa19a206d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:14.794844 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd4464-fca2-43ad-9c3d-1b3aa19a206d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:14.795131 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd4464-fca2-43ad-9c3d-1b3aa19a206d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:14.795406 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd4464-fca2-43ad-9c3d-1b3aa19a206d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:14.806636 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c867e09-a162-4940-8ed9-53a1b062c408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:14.807224 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c867e09-a162-4940-8ed9-53a1b062c408 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:14.807503 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c867e09-a162-4940-8ed9-53a1b062c408 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:14.807776 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c867e09-a162-4940-8ed9-53a1b062c408 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:14.808416 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c867e09-a162-4940-8ed9-53a1b062c408 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:14.808714 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c867e09-a162-4940-8ed9-53a1b062c408 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:14.810056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c867e09-a162-4940-8ed9-53a1b062c408 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:14.821500 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 544/1070] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:19:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:14.822461 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c867e09-a162-4940-8ed9-53a1b062c408 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:14.866244 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd4464-fca2-43ad-9c3d-1b3aa19a206d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:14.867684 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd4464-fca2-43ad-9c3d-1b3aa19a206d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:14.889903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:14.922046 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c867e09-a162-4940-8ed9-53a1b062c408 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:14.950353 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:14.956524 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485eaa50-577d-4716-8d71-1ef10cab8131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:14.957208 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485eaa50-577d-4716-8d71-1ef10cab8131 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:14.957565 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485eaa50-577d-4716-8d71-1ef10cab8131 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:14.957914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485eaa50-577d-4716-8d71-1ef10cab8131 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:14.959182 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485eaa50-577d-4716-8d71-1ef10cab8131 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:14.959562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485eaa50-577d-4716-8d71-1ef10cab8131 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:14.960984 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485eaa50-577d-4716-8d71-1ef10cab8131 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:14.967731 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 527/1071] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:19:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:19:14.968561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd4464-fca2-43ad-9c3d-1b3aa19a206d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:14.974054 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 545/1072] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:19:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:14.974054 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485eaa50-577d-4716-8d71-1ef10cab8131 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:15.088945 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd4464-fca2-43ad-9c3d-1b3aa19a206d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:15.106454 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485eaa50-577d-4716-8d71-1ef10cab8131 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:15.114410 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:15.134132 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5399e4b-2978-4845-b8c6-8997bf267062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:15.134801 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5399e4b-2978-4845-b8c6-8997bf267062 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:15.135088 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5399e4b-2978-4845-b8c6-8997bf267062 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:15.135369 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5399e4b-2978-4845-b8c6-8997bf267062 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:15.143290 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5399e4b-2978-4845-b8c6-8997bf267062 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:15.144850 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5399e4b-2978-4845-b8c6-8997bf267062 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:15.145558 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97e52f-4ada-4b20-a9f8-19996eb978c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:15.146757 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97e52f-4ada-4b20-a9f8-19996eb978c2 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:15.147135 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97e52f-4ada-4b20-a9f8-19996eb978c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:15.147319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97e52f-4ada-4b20-a9f8-19996eb978c2 None admin] PATH_INFO: `/v3/projects/99b85d5359534a7a913a1ca963af82a1` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:15.148497 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d97e52f-4ada-4b20-a9f8-19996eb978c2 None admin] RBAC: Authorizing `identity:get_project(project_id=99b85d5359534a7a913a1ca963af82a1)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:15.149903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d97e52f-4ada-4b20-a9f8-19996eb978c2 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:15.151707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 546/1073] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:19:14 2022] GET /identity/v3/projects/99b85d5359534a7a913a1ca963af82a1 => generated 311 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:15.157640 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97e52f-4ada-4b20-a9f8-19996eb978c2 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:15.158294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-f5399e4b-2978-4845-b8c6-8997bf267062 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:19:15.158294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:15.158294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:15.158294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:15.158294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:15.158294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:15.158294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:15.158294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:15.158294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:15.158294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:15.158294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:15.158294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:15.158294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:19:15.158294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:19:15.158294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:19:15.159189 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:19:15.159189 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:15.159189 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:15.159189 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:15.159189 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:15.159189 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:15.159189 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:15.159189 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:15.159189 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:15.159189 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:15.159189 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:15.159189 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:15.159189 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:15.159189 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:15.159189 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:15.159189 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:15.159189 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:15.160129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:19:15.160129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:19:15.160129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:19:15.160129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:19:15.160129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:19:15.160129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:19:15.160129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:19:15.160129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:15.160129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 528/1074] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:19:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:19:15.167171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5399e4b-2978-4845-b8c6-8997bf267062 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:15.226040 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:15.273028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97e52f-4ada-4b20-a9f8-19996eb978c2 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:15.282257 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5399e4b-2978-4845-b8c6-8997bf267062 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:15.306669 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41994d56-2107-44f1-ae5f-69ad32542f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:15.307190 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41994d56-2107-44f1-ae5f-69ad32542f46 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:15.307463 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41994d56-2107-44f1-ae5f-69ad32542f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:15.307732 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41994d56-2107-44f1-ae5f-69ad32542f46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:15.308445 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41994d56-2107-44f1-ae5f-69ad32542f46 None admin] RBAC: Adding query filter params (role.id=ff293acb8551463aafe6970230042b89, scope.project.id=99b85d5359534a7a913a1ca963af82a1, user.id=c73961187bf449c18c2cc14f438650b9) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:15.308750 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41994d56-2107-44f1-ae5f-69ad32542f46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:15.310160 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41994d56-2107-44f1-ae5f-69ad32542f46 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:15.315349 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4080a904-3888-4800-98a3-c22dee65cd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:15.315480 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 547/1075] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:19:15 2022] GET /identity/v3/role_assignments?user.id=c73961187bf449c18c2cc14f438650b9&scope.project.id=99b85d5359534a7a913a1ca963af82a1&role.id=ff293acb8551463aafe6970230042b89 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:19:15.316039 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080a904-3888-4800-98a3-c22dee65cd99 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:15.316333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080a904-3888-4800-98a3-c22dee65cd99 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:15.316677 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080a904-3888-4800-98a3-c22dee65cd99 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:15.317378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4080a904-3888-4800-98a3-c22dee65cd99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:15.318230 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4080a904-3888-4800-98a3-c22dee65cd99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:15.319684 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4080a904-3888-4800-98a3-c22dee65cd99 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:15.334763 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 529/1076] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:19:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:19:15.341082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41994d56-2107-44f1-ae5f-69ad32542f46 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:15.446973 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41994d56-2107-44f1-ae5f-69ad32542f46 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:15.469699 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:15.481647 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf2aaf-3677-4ac6-bad0-9769edbad95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:15.482400 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf2aaf-3677-4ac6-bad0-9769edbad95e None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:15.482676 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf2aaf-3677-4ac6-bad0-9769edbad95e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:15.482949 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf2aaf-3677-4ac6-bad0-9769edbad95e None admin] PATH_INFO: `/v3/projects/479530ced580426b85d1fc07af43417f/users/1f0c99fdf8bd4331ad7e641ee38554c1/roles/1ff98ccdce644654b43e48dc28fce36b` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:15.485175 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf2aaf-3677-4ac6-bad0-9769edbad95e None admin] RBAC: Authorizing `identity:create_grant(project_id=479530ced580426b85d1fc07af43417f, user_id=1f0c99fdf8bd4331ad7e641ee38554c1, role_id=1ff98ccdce644654b43e48dc28fce36b)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:15.486626 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf2aaf-3677-4ac6-bad0-9769edbad95e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:15.500693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 548/1077] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Sep 1 16:19:15 2022] PUT /identity/v3/projects/479530ced580426b85d1fc07af43417f/users/1f0c99fdf8bd4331ad7e641ee38554c1/roles/1ff98ccdce644654b43e48dc28fce36b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:19:17.215629 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413ef09-0303-4820-adf0-aaf7bf94aab0 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:17.216150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413ef09-0303-4820-adf0-aaf7bf94aab0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:17.216242 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413ef09-0303-4820-adf0-aaf7bf94aab0 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:17.217665 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 530/1078] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:19:17.222660 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5dede-f989-4c1c-922d-d519a5c1e056 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:17.223104 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5dede-f989-4c1c-922d-d519a5c1e056 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:17.223201 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5dede-f989-4c1c-922d-d519a5c1e056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:17.241614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-4be5dede-f989-4c1c-922d-d519a5c1e056 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:17.242851 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4be5dede-f989-4c1c-922d-d519a5c1e056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:17.308734 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 549/1079] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:19:17.406219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291e26c-d7fa-44df-93fa-cce98c283a65 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:17.406507 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291e26c-d7fa-44df-93fa-cce98c283a65 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:17.406787 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291e26c-d7fa-44df-93fa-cce98c283a65 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:17.408136 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 531/1080] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:19:17.413809 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca2fdd-133c-419d-9a40-7fc73f1be849 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:17.414096 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca2fdd-133c-419d-9a40-7fc73f1be849 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:17.414411 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca2fdd-133c-419d-9a40-7fc73f1be849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:17.432525 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-1bca2fdd-133c-419d-9a40-7fc73f1be849 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:17.433707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1bca2fdd-133c-419d-9a40-7fc73f1be849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:17.499575 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 550/1081] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:19:17.572980 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291e26c-d7fa-44df-93fa-cce98c283a65 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:17.616355 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:17.674039 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291e26c-d7fa-44df-93fa-cce98c283a65 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:17.706010 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea1bed-af8a-4e94-995e-502c8f79811c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:17.706654 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea1bed-af8a-4e94-995e-502c8f79811c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:17.706935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea1bed-af8a-4e94-995e-502c8f79811c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:17.707205 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea1bed-af8a-4e94-995e-502c8f79811c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:17.717731 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea1bed-af8a-4e94-995e-502c8f79811c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:19:17.722463 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-d3ea1bed-af8a-4e94-995e-502c8f79811c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:19:17.722463 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:17.722463 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:17.722463 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:17.722463 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:17.722463 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:17.722463 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:17.722463 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:17.722463 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:17.722463 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:17.722463 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:17.722463 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:17.722463 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:19:17.722463 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:19:17.722463 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:19:17.722463 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:19:17.723597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:19:17.723597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:19:17.723597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:19:17.723597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:19:17.723597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:17.723597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:17.723597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:17.723597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:17.723597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:17.723597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:17.723597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:17.723597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:17.723597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:17.723597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:17.723597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:17.723597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:17.724485 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:17.724485 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:17.724485 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:17.724485 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:17.724485 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:19:17.724485 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:19:17.724485 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:19:17.724485 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:19:17.724485 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:19:17.724485 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:19:17.724485 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:19:17.724485 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:17.724485 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 532/1082] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:19:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:19:17.729405 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca2fdd-133c-419d-9a40-7fc73f1be849 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:17.771003 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:17.776603 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea1bed-af8a-4e94-995e-502c8f79811c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:17.827910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca2fdd-133c-419d-9a40-7fc73f1be849 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:17.860824 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f5a46-1179-45f6-a0c9-bcbc4ea9be82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:17.861418 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f5a46-1179-45f6-a0c9-bcbc4ea9be82 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:17.861693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f5a46-1179-45f6-a0c9-bcbc4ea9be82 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:17.861963 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f5a46-1179-45f6-a0c9-bcbc4ea9be82 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:17.863064 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0f5a46-1179-45f6-a0c9-bcbc4ea9be82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:17.863373 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0f5a46-1179-45f6-a0c9-bcbc4ea9be82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:17.865242 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0f5a46-1179-45f6-a0c9-bcbc4ea9be82 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:17.872958 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 551/1083] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:19:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:17.874246 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea1bed-af8a-4e94-995e-502c8f79811c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:17.875810 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:17.878677 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f5a46-1179-45f6-a0c9-bcbc4ea9be82 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:17.909075 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a8d06-2fb7-4c0e-9bcd-2391dde9bf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:17.909701 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a8d06-2fb7-4c0e-9bcd-2391dde9bf0d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:17.909981 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a8d06-2fb7-4c0e-9bcd-2391dde9bf0d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:17.910277 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a8d06-2fb7-4c0e-9bcd-2391dde9bf0d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:17.920368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a8d06-2fb7-4c0e-9bcd-2391dde9bf0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:19:17.925110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-6a1a8d06-2fb7-4c0e-9bcd-2391dde9bf0d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:19:17.925110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:17.925110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:17.925110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:17.925110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:17.925110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:17.925110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:17.925110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:17.925110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:17.925110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:17.925110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:17.925110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:17.925110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:19:17.925110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:19:17.925110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:19:17.925110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:19:17.926048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:19:17.926048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:19:17.926048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:19:17.926048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:19:17.926048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:17.926048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:17.926048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:17.926048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:17.926048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:17.926048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:17.926048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:17.926048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:17.926048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:17.926048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:17.926048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:17.926048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:17.927491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:17.927491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:17.927491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:17.927491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:17.927491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:19:17.927491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:19:17.927491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:19:17.927491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:19:17.927491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:19:17.927491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:19:17.927491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:19:17.927491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:17.927491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 533/1084] 10.222.0.58 () {56 vars in 1120 bytes} [Thu Sep 1 16:19:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:19:17.932497 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a8d06-2fb7-4c0e-9bcd-2391dde9bf0d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:17.976059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f5a46-1179-45f6-a0c9-bcbc4ea9be82 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:17.978763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:18.009756 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f7a31-63a8-42ae-b96d-227ec93b6c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:18.010408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f7a31-63a8-42ae-b96d-227ec93b6c59 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:18.010691 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f7a31-63a8-42ae-b96d-227ec93b6c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:18.010965 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f7a31-63a8-42ae-b96d-227ec93b6c59 None admin] PATH_INFO: `/v3/users/c73961187bf449c18c2cc14f438650b9` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:18.012113 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f7a31-63a8-42ae-b96d-227ec93b6c59 None admin] RBAC: Authorizing `identity:get_user(user_id=c73961187bf449c18c2cc14f438650b9)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:18.013536 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f7a31-63a8-42ae-b96d-227ec93b6c59 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:18.015134 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 552/1085] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:19:17 2022] GET /identity/v3/users/c73961187bf449c18c2cc14f438650b9 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:19:18.020973 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f7a31-63a8-42ae-b96d-227ec93b6c59 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:18.029928 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a8d06-2fb7-4c0e-9bcd-2391dde9bf0d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:18.067356 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f3413-f84c-442b-997a-f9913c714e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:18.068547 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f3413-f84c-442b-997a-f9913c714e8c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:18.068829 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f3413-f84c-442b-997a-f9913c714e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:18.069105 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f3413-f84c-442b-997a-f9913c714e8c None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:18.069776 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f3413-f84c-442b-997a-f9913c714e8c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:18.070085 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f3413-f84c-442b-997a-f9913c714e8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:18.071490 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f3413-f84c-442b-997a-f9913c714e8c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:18.079597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 534/1086] 10.222.0.58 () {56 vars in 1115 bytes} [Thu Sep 1 16:19:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:19:18.085094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f3413-f84c-442b-997a-f9913c714e8c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:18.108740 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:18.120939 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f7a31-63a8-42ae-b96d-227ec93b6c59 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:18.154401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1c63f-205e-42bb-adaa-862d907eb41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:18.155096 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1c63f-205e-42bb-adaa-862d907eb41d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:18.155379 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1c63f-205e-42bb-adaa-862d907eb41d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:18.155656 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1c63f-205e-42bb-adaa-862d907eb41d None admin] PATH_INFO: `/v3/projects/99b85d5359534a7a913a1ca963af82a1` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:18.156767 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b1c63f-205e-42bb-adaa-862d907eb41d None admin] RBAC: Authorizing `identity:get_project(project_id=99b85d5359534a7a913a1ca963af82a1)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:18.158361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b1c63f-205e-42bb-adaa-862d907eb41d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:18.159982 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 553/1087] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:19:18 2022] GET /identity/v3/projects/99b85d5359534a7a913a1ca963af82a1 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:19:18.167143 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1c63f-205e-42bb-adaa-862d907eb41d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:18.183971 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f3413-f84c-442b-997a-f9913c714e8c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:18.207394 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:18.219086 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705af8f-aab7-4a46-af7e-1904e33a27e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:18.220516 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705af8f-aab7-4a46-af7e-1904e33a27e8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:18.220690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705af8f-aab7-4a46-af7e-1904e33a27e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:18.221199 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705af8f-aab7-4a46-af7e-1904e33a27e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:18.233102 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b705af8f-aab7-4a46-af7e-1904e33a27e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:18.234591 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b705af8f-aab7-4a46-af7e-1904e33a27e8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:18.249141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-b705af8f-aab7-4a46-af7e-1904e33a27e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:19:18.249141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:18.249141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:18.249141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:18.249141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:18.249141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:18.249141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:18.249141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:18.249141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:18.249141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:18.249141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:18.249141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:18.249141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:19:18.249141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:19:18.249141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:19:18.249141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:19:18.250679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:18.250679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:18.250679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:18.250679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:18.250679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:18.250679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:18.250679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:18.250679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:18.250679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:18.250679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:18.250679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:18.250679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:18.250679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:18.250679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:18.250679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:18.250679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:18.250679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:19:18.251616 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:19:18.251616 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:19:18.251616 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:18.251616 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 535/1088] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:19:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:19:18.256801 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705af8f-aab7-4a46-af7e-1904e33a27e8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:18.272755 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1c63f-205e-42bb-adaa-862d907eb41d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:18.307926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19d846-e996-41fc-be40-e5a5d8664317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:18.308579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19d846-e996-41fc-be40-e5a5d8664317 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:18.308854 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19d846-e996-41fc-be40-e5a5d8664317 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:18.309127 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19d846-e996-41fc-be40-e5a5d8664317 None admin] PATH_INFO: `/v3/projects/99b85d5359534a7a913a1ca963af82a1/users/c73961187bf449c18c2cc14f438650b9/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:18.311046 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b19d846-e996-41fc-be40-e5a5d8664317 None admin] RBAC: Authorizing `identity:create_grant(project_id=99b85d5359534a7a913a1ca963af82a1, user_id=c73961187bf449c18c2cc14f438650b9, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:18.312457 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b19d846-e996-41fc-be40-e5a5d8664317 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:18.326660 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 554/1089] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Sep 1 16:19:18 2022] PUT /identity/v3/projects/99b85d5359534a7a913a1ca963af82a1/users/c73961187bf449c18c2cc14f438650b9/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:19:18.339273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:18.365003 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705af8f-aab7-4a46-af7e-1904e33a27e8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:18.401946 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab0406-cf10-4ad1-8549-2f2b49bd0dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:18.402600 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab0406-cf10-4ad1-8549-2f2b49bd0dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:18.402882 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab0406-cf10-4ad1-8549-2f2b49bd0dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:18.403158 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab0406-cf10-4ad1-8549-2f2b49bd0dc3 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:18.403815 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ab0406-cf10-4ad1-8549-2f2b49bd0dc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:18.404121 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ab0406-cf10-4ad1-8549-2f2b49bd0dc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:18.405510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ab0406-cf10-4ad1-8549-2f2b49bd0dc3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:18.418442 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 536/1090] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:19:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:19:18.424387 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19d846-e996-41fc-be40-e5a5d8664317 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:18.527963 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:18.531418 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19d846-e996-41fc-be40-e5a5d8664317 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:18.566674 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051b71e-16b2-4291-93fd-448d1a169519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:18.567307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051b71e-16b2-4291-93fd-448d1a169519 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:18.568081 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051b71e-16b2-4291-93fd-448d1a169519 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:18.569220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051b71e-16b2-4291-93fd-448d1a169519 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:18.576680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b051b71e-16b2-4291-93fd-448d1a169519 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:18.578104 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b051b71e-16b2-4291-93fd-448d1a169519 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:18.589550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-b051b71e-16b2-4291-93fd-448d1a169519 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:19:18.589550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:18.589550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:18.589550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:18.589550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:18.589550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:18.589550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:18.589550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:18.589550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:18.589550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:18.589550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:18.589550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:18.589550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:19:18.589550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:19:18.589550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:19:18.589550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:19:18.590580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:18.590580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:18.590580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:19:18.590580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:19:18.590580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:19:18.590580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:19:18.590580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:18.590580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:18.590580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:18.590580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:18.590580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:18.590580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:18.590580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:18.590580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:18.590580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:18.590580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:18.590580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:18.590580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:18.591608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:18.591608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:18.591608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:19:18.591608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:19:18.591608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:19:18.591608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:19:18.591608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:19:18.591608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:19:18.591608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:19:18.591608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:19:18.591608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 555/1091] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Sep 1 16:19:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:19:18.596319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab0406-cf10-4ad1-8549-2f2b49bd0dc3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:18.696043 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab0406-cf10-4ad1-8549-2f2b49bd0dc3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:18.727513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1767d-f993-4034-a2dd-b4b15fa50818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:18.728106 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1767d-f993-4034-a2dd-b4b15fa50818 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:18.728383 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1767d-f993-4034-a2dd-b4b15fa50818 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:18.728651 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1767d-f993-4034-a2dd-b4b15fa50818 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:18.729365 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1767d-f993-4034-a2dd-b4b15fa50818 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:18.729668 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1767d-f993-4034-a2dd-b4b15fa50818 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:18.731102 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1767d-f993-4034-a2dd-b4b15fa50818 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:18.733653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:18.749629 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 537/1092] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Sep 1 16:19:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:18.756023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051b71e-16b2-4291-93fd-448d1a169519 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:18.852429 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051b71e-16b2-4291-93fd-448d1a169519 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:18.884736 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1480ab2a-0a4b-4d5d-916d-79f79d18c4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:18.885323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480ab2a-0a4b-4d5d-916d-79f79d18c4b0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:18.885598 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480ab2a-0a4b-4d5d-916d-79f79d18c4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:18.885866 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480ab2a-0a4b-4d5d-916d-79f79d18c4b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:18.911574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:18.951058 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1480ab2a-0a4b-4d5d-916d-79f79d18c4b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:18.952488 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1480ab2a-0a4b-4d5d-916d-79f79d18c4b0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:19.050530 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 556/1093] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:19:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:19:19.052998 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1767d-f993-4034-a2dd-b4b15fa50818 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:19.152025 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1767d-f993-4034-a2dd-b4b15fa50818 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:19.185729 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37913ca9-1b0b-4148-9a98-54fb40b24281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:19.186386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37913ca9-1b0b-4148-9a98-54fb40b24281 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:19.186669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37913ca9-1b0b-4148-9a98-54fb40b24281 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:19.186943 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37913ca9-1b0b-4148-9a98-54fb40b24281 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:19.196546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37913ca9-1b0b-4148-9a98-54fb40b24281 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:19.197990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37913ca9-1b0b-4148-9a98-54fb40b24281 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:19.211915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-37913ca9-1b0b-4148-9a98-54fb40b24281 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:19:19.211915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:19.211915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:19.211915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:19.211915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:19.211915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:19.211915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:19.211915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:19.211915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:19.211915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:19.211915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:19.211915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:19.211915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:19:19.211915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:19:19.211915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:19:19.211915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:19:19.212862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:19.212862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:19.212862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:19.212862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:19.212862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:19.212862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:19.212862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:19.212862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:19.212862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:19.212862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:19.212862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:19.212862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:19.212862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:19.212862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:19.212862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:19.212862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:19.212862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:19:19.213784 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:19:19.213784 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:19:19.213784 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:19.213784 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 538/1094] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:19:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:19:19.218836 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1480ab2a-0a4b-4d5d-916d-79f79d18c4b0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:19.276723 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:19.317123 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1480ab2a-0a4b-4d5d-916d-79f79d18c4b0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:19.351067 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf172a0-8cc2-44b1-9c10-c30edeced398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:19.351792 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf172a0-8cc2-44b1-9c10-c30edeced398 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:19.352068 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf172a0-8cc2-44b1-9c10-c30edeced398 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:19.352335 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf172a0-8cc2-44b1-9c10-c30edeced398 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:19.352969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf172a0-8cc2-44b1-9c10-c30edeced398 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:19.353265 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf172a0-8cc2-44b1-9c10-c30edeced398 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:19.354642 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf172a0-8cc2-44b1-9c10-c30edeced398 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:19.366342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 557/1095] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:19:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:19.371934 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37913ca9-1b0b-4148-9a98-54fb40b24281 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:19.419418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:19.471002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37913ca9-1b0b-4148-9a98-54fb40b24281 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:19.502720 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4c5ce-3b23-4d74-b23c-52c363152e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:19.503336 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4c5ce-3b23-4d74-b23c-52c363152e94 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:19.503611 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4c5ce-3b23-4d74-b23c-52c363152e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:19.503879 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4c5ce-3b23-4d74-b23c-52c363152e94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:19.510724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4c5ce-3b23-4d74-b23c-52c363152e94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:19.512148 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4c5ce-3b23-4d74-b23c-52c363152e94 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:19.522896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-ddd4c5ce-3b23-4d74-b23c-52c363152e94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:19:19.522896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:19.522896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:19.522896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:19.522896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:19.522896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:19.522896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:19.522896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:19.522896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:19.522896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:19.522896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:19.522896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:19.522896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:19:19.522896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:19:19.522896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:19:19.523927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:19:19.523927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:19.523927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:19.523927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:19.523927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:19.523927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:19.523927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:19.523927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:19.523927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:19.523927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:19.523927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:19.523927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:19.523927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:19.523927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:19.523927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:19.523927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:19.523927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:19.525094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:19:19.525094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:19:19.525094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:19:19.525094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:19:19.525094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:19:19.525094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:19:19.525094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:19:19.525094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:19.525094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 539/1096] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:19:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:19:19.529795 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf172a0-8cc2-44b1-9c10-c30edeced398 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:19.625123 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf172a0-8cc2-44b1-9c10-c30edeced398 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:19.644537 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:19.661986 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b45e4-a8be-4d7a-9048-d6fc0b5d8ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:19.662685 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b45e4-a8be-4d7a-9048-d6fc0b5d8ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:19.662960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b45e4-a8be-4d7a-9048-d6fc0b5d8ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:19.663227 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b45e4-a8be-4d7a-9048-d6fc0b5d8ea3 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:19.663913 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46b45e4-a8be-4d7a-9048-d6fc0b5d8ea3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:19.664212 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46b45e4-a8be-4d7a-9048-d6fc0b5d8ea3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:19.665614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46b45e4-a8be-4d7a-9048-d6fc0b5d8ea3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:19.683190 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 558/1097] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:19:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:19:19.684251 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4c5ce-3b23-4d74-b23c-52c363152e94 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:19.782125 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4c5ce-3b23-4d74-b23c-52c363152e94 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:19.789508 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:19.817112 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0544b13-9294-4132-a712-5147b67310d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:19.817802 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0544b13-9294-4132-a712-5147b67310d9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:19.818176 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0544b13-9294-4132-a712-5147b67310d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:19.818491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0544b13-9294-4132-a712-5147b67310d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:19.819227 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0544b13-9294-4132-a712-5147b67310d9 None admin] RBAC: Adding query filter params (role.id=1ff98ccdce644654b43e48dc28fce36b, scope.project.id=479530ced580426b85d1fc07af43417f, user.id=1f0c99fdf8bd4331ad7e641ee38554c1) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:19.819538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0544b13-9294-4132-a712-5147b67310d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:19.820961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0544b13-9294-4132-a712-5147b67310d9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:19.826819 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 540/1098] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:19:19 2022] GET /identity/v3/role_assignments?user.id=1f0c99fdf8bd4331ad7e641ee38554c1&scope.project.id=479530ced580426b85d1fc07af43417f&role.id=1ff98ccdce644654b43e48dc28fce36b => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:20.242479 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237c753-ee85-4b75-a343-f2ded39788bf None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:20.242759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237c753-ee85-4b75-a343-f2ded39788bf None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:20.243031 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237c753-ee85-4b75-a343-f2ded39788bf None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:20.244382 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 559/1099] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:19:20.249648 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248dbdc-0988-465a-bce3-7fdedb9ebb3c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:20.249940 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248dbdc-0988-465a-bce3-7fdedb9ebb3c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:20.250258 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248dbdc-0988-465a-bce3-7fdedb9ebb3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:20.268933 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2248dbdc-0988-465a-bce3-7fdedb9ebb3c None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:20.270137 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2248dbdc-0988-465a-bce3-7fdedb9ebb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:20.341739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 541/1100] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:19:20.609074 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237c753-ee85-4b75-a343-f2ded39788bf None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:20.713017 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237c753-ee85-4b75-a343-f2ded39788bf None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:20.746122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440880d-5f62-4561-a160-da2bbcc04ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:20.746757 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440880d-5f62-4561-a160-da2bbcc04ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:20.747035 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440880d-5f62-4561-a160-da2bbcc04ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:20.747309 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440880d-5f62-4561-a160-da2bbcc04ec2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:20.756973 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9440880d-5f62-4561-a160-da2bbcc04ec2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:19:20.761668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-9440880d-5f62-4561-a160-da2bbcc04ec2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:19:20.761668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:20.761668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:20.761668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:20.761668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:20.761668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:20.761668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:20.761668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:20.761668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:20.761668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:20.761668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:20.761668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:20.761668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:19:20.761668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:19:20.761668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:19:20.761668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:19:20.762787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:19:20.762787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:19:20.762787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:19:20.762787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:19:20.762787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:20.762787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:20.762787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:20.762787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:20.762787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:20.762787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:20.762787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:20.762787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:20.762787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:20.762787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:20.762787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:20.762787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:20.764023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:20.764023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:20.764023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:20.764023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:20.764023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:19:20.764023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:19:20.764023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:19:20.764023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:19:20.764023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:19:20.764023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:19:20.764023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:19:20.764023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:19:20.764023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 560/1101] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:19:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:19:20.768895 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248dbdc-0988-465a-bce3-7fdedb9ebb3c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:20.861599 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:20.864531 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248dbdc-0988-465a-bce3-7fdedb9ebb3c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:20.903216 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53a425-3066-4232-a185-ec0dc4bf7946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:20.903818 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53a425-3066-4232-a185-ec0dc4bf7946 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:20.904100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53a425-3066-4232-a185-ec0dc4bf7946 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:20.904376 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53a425-3066-4232-a185-ec0dc4bf7946 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:20.905133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53a425-3066-4232-a185-ec0dc4bf7946 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:20.905425 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53a425-3066-4232-a185-ec0dc4bf7946 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:20.906815 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53a425-3066-4232-a185-ec0dc4bf7946 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:20.915388 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 542/1102] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:19:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:20.921369 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440880d-5f62-4561-a160-da2bbcc04ec2 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:20.962575 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:21.020123 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440880d-5f62-4561-a160-da2bbcc04ec2 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:21.053604 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1242d8a-fbc8-4699-bc38-1a31aadddbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:21.054254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1242d8a-fbc8-4699-bc38-1a31aadddbff None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:21.054540 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1242d8a-fbc8-4699-bc38-1a31aadddbff None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:21.054814 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1242d8a-fbc8-4699-bc38-1a31aadddbff None admin] PATH_INFO: `/v3/users/c73961187bf449c18c2cc14f438650b9` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:21.055921 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1242d8a-fbc8-4699-bc38-1a31aadddbff None admin] RBAC: Authorizing `identity:get_user(user_id=c73961187bf449c18c2cc14f438650b9)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:21.057344 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1242d8a-fbc8-4699-bc38-1a31aadddbff None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:21.058946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 561/1103] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:19:20 2022] GET /identity/v3/users/c73961187bf449c18c2cc14f438650b9 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:21.064861 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53a425-3066-4232-a185-ec0dc4bf7946 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:21.161647 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53a425-3066-4232-a185-ec0dc4bf7946 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:21.194739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1cc69e-fe18-4be1-81bb-bbbf8e2e395e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:21.195329 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cc69e-fe18-4be1-81bb-bbbf8e2e395e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:21.195620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cc69e-fe18-4be1-81bb-bbbf8e2e395e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:21.195896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cc69e-fe18-4be1-81bb-bbbf8e2e395e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:21.225086 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:21.261976 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1cc69e-fe18-4be1-81bb-bbbf8e2e395e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:21.263429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1cc69e-fe18-4be1-81bb-bbbf8e2e395e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:21.356865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 543/1104] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:19:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:19:21.362934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1242d8a-fbc8-4699-bc38-1a31aadddbff None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:21.459784 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1242d8a-fbc8-4699-bc38-1a31aadddbff None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:21.468603 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:21.493062 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d4cd5-79fd-47a4-9af8-cbfc3f59dd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:21.493686 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d4cd5-79fd-47a4-9af8-cbfc3f59dd0f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:21.493967 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d4cd5-79fd-47a4-9af8-cbfc3f59dd0f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:21.494273 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d4cd5-79fd-47a4-9af8-cbfc3f59dd0f None admin] PATH_INFO: `/v3/projects/99b85d5359534a7a913a1ca963af82a1` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:21.495484 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544d4cd5-79fd-47a4-9af8-cbfc3f59dd0f None admin] RBAC: Authorizing `identity:get_project(project_id=99b85d5359534a7a913a1ca963af82a1)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:21.496927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544d4cd5-79fd-47a4-9af8-cbfc3f59dd0f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:21.498712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 562/1105] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:19:21 2022] GET /identity/v3/projects/99b85d5359534a7a913a1ca963af82a1 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:21.505476 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1cc69e-fe18-4be1-81bb-bbbf8e2e395e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:21.564601 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:21.602786 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1cc69e-fe18-4be1-81bb-bbbf8e2e395e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:21.637596 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6bacc1-817d-4993-abf4-f290cc981463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:21.638186 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bacc1-817d-4993-abf4-f290cc981463 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:21.638501 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bacc1-817d-4993-abf4-f290cc981463 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:21.638776 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bacc1-817d-4993-abf4-f290cc981463 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:21.639504 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6bacc1-817d-4993-abf4-f290cc981463 None admin] RBAC: Adding query filter params (role.id=ff293acb8551463aafe6970230042b89, scope.project.id=99b85d5359534a7a913a1ca963af82a1, user.id=c73961187bf449c18c2cc14f438650b9) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:21.639815 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6bacc1-817d-4993-abf4-f290cc981463 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:21.641228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6bacc1-817d-4993-abf4-f290cc981463 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:21.647347 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 544/1106] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:19:21 2022] GET /identity/v3/role_assignments?user.id=c73961187bf449c18c2cc14f438650b9&scope.project.id=99b85d5359534a7a913a1ca963af82a1&role.id=ff293acb8551463aafe6970230042b89 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:23.566164 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09669ee-db67-4d7f-ade8-b3cff8a56364 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:23.566770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09669ee-db67-4d7f-ade8-b3cff8a56364 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:23.566770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09669ee-db67-4d7f-ade8-b3cff8a56364 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:23.568560 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 563/1107] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:19:23.573533 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe96841-246d-477b-b47a-0f552f90b000 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:23.573940 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe96841-246d-477b-b47a-0f552f90b000 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:23.574081 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe96841-246d-477b-b47a-0f552f90b000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:23.592342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1fe96841-246d-477b-b47a-0f552f90b000 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:23.593517 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1fe96841-246d-477b-b47a-0f552f90b000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:23.659817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 545/1108] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:19:23.926292 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09669ee-db67-4d7f-ade8-b3cff8a56364 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:24.024352 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09669ee-db67-4d7f-ade8-b3cff8a56364 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:24.057945 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3226670-433f-47cb-8126-98df34e55284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:24.058603 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3226670-433f-47cb-8126-98df34e55284 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:24.058887 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3226670-433f-47cb-8126-98df34e55284 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:24.059162 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3226670-433f-47cb-8126-98df34e55284 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:24.060291 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3226670-433f-47cb-8126-98df34e55284 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:24.061760 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3226670-433f-47cb-8126-98df34e55284 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:24.063390 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 564/1109] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:19:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:24.069075 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe96841-246d-477b-b47a-0f552f90b000 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:24.123313 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:24.165364 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe96841-246d-477b-b47a-0f552f90b000 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:24.198872 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a91ad5-2271-4358-a53d-0465fc9909ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:24.199418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a91ad5-2271-4358-a53d-0465fc9909ea None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:24.199701 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a91ad5-2271-4358-a53d-0465fc9909ea None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:24.199975 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a91ad5-2271-4358-a53d-0465fc9909ea None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:24.200613 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a91ad5-2271-4358-a53d-0465fc9909ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:24.202042 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a91ad5-2271-4358-a53d-0465fc9909ea None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:24.221978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 546/1110] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Sep 1 16:19:24 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:19:26.186034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e0e71-eddf-42f3-abf1-d96999dcf839 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:26.186626 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e0e71-eddf-42f3-abf1-d96999dcf839 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:26.186626 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e0e71-eddf-42f3-abf1-d96999dcf839 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:26.187910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 565/1111] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:19:26.193266 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bcbf0-4242-4dea-84d7-e19cf0c3f37b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:26.193658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bcbf0-4242-4dea-84d7-e19cf0c3f37b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:26.193828 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bcbf0-4242-4dea-84d7-e19cf0c3f37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:26.212081 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7a1bcbf0-4242-4dea-84d7-e19cf0c3f37b None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:26.213268 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7a1bcbf0-4242-4dea-84d7-e19cf0c3f37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:26.285454 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 547/1112] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:19:26.559229 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e0e71-eddf-42f3-abf1-d96999dcf839 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:26.614933 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:26.668775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e0e71-eddf-42f3-abf1-d96999dcf839 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:26.701329 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dfe698-dd90-40e0-b7c1-163c8e8ca948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:26.701939 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dfe698-dd90-40e0-b7c1-163c8e8ca948 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:26.702241 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dfe698-dd90-40e0-b7c1-163c8e8ca948 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:26.702518 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dfe698-dd90-40e0-b7c1-163c8e8ca948 None admin] PATH_INFO: `/v3/roles/72700efc336a4b4f992b08ff9ea99f0c` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:26.703900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dfe698-dd90-40e0-b7c1-163c8e8ca948 None admin] RBAC: Authorizing `identity:get_role(role_id=72700efc336a4b4f992b08ff9ea99f0c)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:26.705252 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dfe698-dd90-40e0-b7c1-163c8e8ca948 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:26.706580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 566/1113] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:19:26 2022] GET /identity/v3/roles/72700efc336a4b4f992b08ff9ea99f0c => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:19:26.713301 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1bcbf0-4242-4dea-84d7-e19cf0c3f37b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:26.753999 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:26.811149 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1bcbf0-4242-4dea-84d7-e19cf0c3f37b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:26.843183 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa2b5d-e9d3-4c26-99d2-17e7d9877ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:26.843800 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa2b5d-e9d3-4c26-99d2-17e7d9877ded None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:26.844091 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa2b5d-e9d3-4c26-99d2-17e7d9877ded None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:26.844368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa2b5d-e9d3-4c26-99d2-17e7d9877ded None admin] PATH_INFO: `/v3/users/0bc07d0ca9654925b8d95d4cd2f3fc11` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:26.857440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caa2b5d-e9d3-4c26-99d2-17e7d9877ded None admin] RBAC: Authorizing `identity:get_user(user_id=0bc07d0ca9654925b8d95d4cd2f3fc11)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:26.858901 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caa2b5d-e9d3-4c26-99d2-17e7d9877ded None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:26.860482 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 548/1114] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:19:26 2022] GET /identity/v3/users/0bc07d0ca9654925b8d95d4cd2f3fc11 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:26.866834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dfe698-dd90-40e0-b7c1-163c8e8ca948 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:26.970918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dfe698-dd90-40e0-b7c1-163c8e8ca948 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:26.984425 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:27.005224 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c4c67-ca66-42a9-aa99-3e7e5aeaea47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:27.005822 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c4c67-ca66-42a9-aa99-3e7e5aeaea47 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:27.006096 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c4c67-ca66-42a9-aa99-3e7e5aeaea47 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:27.006393 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c4c67-ca66-42a9-aa99-3e7e5aeaea47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:27.070645 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c4c67-ca66-42a9-aa99-3e7e5aeaea47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:27.072016 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c4c67-ca66-42a9-aa99-3e7e5aeaea47 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:27.167300 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 567/1115] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:19:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:19:27.173329 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa2b5d-e9d3-4c26-99d2-17e7d9877ded None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:27.276357 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:27.279592 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa2b5d-e9d3-4c26-99d2-17e7d9877ded None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:27.316242 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd8aab-9289-490d-aa32-283946f68084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:27.316871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd8aab-9289-490d-aa32-283946f68084 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:27.317169 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd8aab-9289-490d-aa32-283946f68084 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:27.317427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd8aab-9289-490d-aa32-283946f68084 None admin] PATH_INFO: `/v3/projects/99b85d5359534a7a913a1ca963af82a1` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:27.318752 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd8aab-9289-490d-aa32-283946f68084 None admin] RBAC: Authorizing `identity:get_project(project_id=99b85d5359534a7a913a1ca963af82a1)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:27.320529 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd8aab-9289-490d-aa32-283946f68084 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:27.322249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 549/1116] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:19:27 2022] GET /identity/v3/projects/99b85d5359534a7a913a1ca963af82a1 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:27.329265 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c4c67-ca66-42a9-aa99-3e7e5aeaea47 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:27.343204 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:27.426248 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c4c67-ca66-42a9-aa99-3e7e5aeaea47 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:27.460286 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6328eb8-82a3-43a1-8247-682a72401a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:27.460876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6328eb8-82a3-43a1-8247-682a72401a98 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:27.461155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6328eb8-82a3-43a1-8247-682a72401a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:27.461427 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6328eb8-82a3-43a1-8247-682a72401a98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:27.462150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6328eb8-82a3-43a1-8247-682a72401a98 None admin] RBAC: Adding query filter params (role.id=72700efc336a4b4f992b08ff9ea99f0c, scope.project.id=99b85d5359534a7a913a1ca963af82a1, user.id=0bc07d0ca9654925b8d95d4cd2f3fc11) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:27.462495 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6328eb8-82a3-43a1-8247-682a72401a98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:27.463914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6328eb8-82a3-43a1-8247-682a72401a98 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:27.469104 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 568/1117] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:19:27 2022] GET /identity/v3/role_assignments?user.id=0bc07d0ca9654925b8d95d4cd2f3fc11&scope.project.id=99b85d5359534a7a913a1ca963af82a1&role.id=72700efc336a4b4f992b08ff9ea99f0c => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:29.388645 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6ef0f-5aa7-4de4-bf91-7094584c1bf6 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:29.389161 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6ef0f-5aa7-4de4-bf91-7094584c1bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:29.389304 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6ef0f-5aa7-4de4-bf91-7094584c1bf6 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:29.390748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 550/1118] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:19:29.396234 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065545d9-69fe-4433-8a0b-c60ca0317a37 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:29.396702 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065545d9-69fe-4433-8a0b-c60ca0317a37 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:29.396795 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065545d9-69fe-4433-8a0b-c60ca0317a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:29.415057 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-065545d9-69fe-4433-8a0b-c60ca0317a37 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:29.416482 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-065545d9-69fe-4433-8a0b-c60ca0317a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:29.481444 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 569/1119] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:19:29.753028 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6ef0f-5aa7-4de4-bf91-7094584c1bf6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:29.853639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6ef0f-5aa7-4de4-bf91-7094584c1bf6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:29.886035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3eab01-86f7-469b-ba0c-d70560e544d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:29.886678 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3eab01-86f7-469b-ba0c-d70560e544d4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:29.886961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3eab01-86f7-469b-ba0c-d70560e544d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:29.887233 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3eab01-86f7-469b-ba0c-d70560e544d4 None admin] PATH_INFO: `/v3/roles/72700efc336a4b4f992b08ff9ea99f0c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:29.888597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3eab01-86f7-469b-ba0c-d70560e544d4 None admin] RBAC: Authorizing `identity:get_role(role_id=72700efc336a4b4f992b08ff9ea99f0c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:29.889951 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3eab01-86f7-469b-ba0c-d70560e544d4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:29.891274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 551/1120] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:19:29 2022] GET /identity/v3/roles/72700efc336a4b4f992b08ff9ea99f0c => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:29.897567 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065545d9-69fe-4433-8a0b-c60ca0317a37 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:29.905589 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:29.993469 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065545d9-69fe-4433-8a0b-c60ca0317a37 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:30.026059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540bba3-5d56-4adc-bd11-f38af6963887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:30.026643 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540bba3-5d56-4adc-bd11-f38af6963887 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:30.026921 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540bba3-5d56-4adc-bd11-f38af6963887 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:30.027193 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540bba3-5d56-4adc-bd11-f38af6963887 None admin] PATH_INFO: `/v3/users/0bc07d0ca9654925b8d95d4cd2f3fc11` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:30.028298 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5540bba3-5d56-4adc-bd11-f38af6963887 None admin] RBAC: Authorizing `identity:get_user(user_id=0bc07d0ca9654925b8d95d4cd2f3fc11)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:30.029767 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5540bba3-5d56-4adc-bd11-f38af6963887 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:30.031326 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 570/1121] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:19:29 2022] GET /identity/v3/users/0bc07d0ca9654925b8d95d4cd2f3fc11 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:30.036967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3eab01-86f7-469b-ba0c-d70560e544d4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:30.071767 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:30.134743 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3eab01-86f7-469b-ba0c-d70560e544d4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:30.168265 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d733aa-bde2-4656-82bf-7ec5505681f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:30.168879 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d733aa-bde2-4656-82bf-7ec5505681f7 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:30.169159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d733aa-bde2-4656-82bf-7ec5505681f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:30.169428 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d733aa-bde2-4656-82bf-7ec5505681f7 None admin] PATH_INFO: `/v3/projects/99b85d5359534a7a913a1ca963af82a1` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:30.170642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d733aa-bde2-4656-82bf-7ec5505681f7 None admin] RBAC: Authorizing `identity:get_project(project_id=99b85d5359534a7a913a1ca963af82a1)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:30.172057 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d733aa-bde2-4656-82bf-7ec5505681f7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:30.173669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 552/1122] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:19:30 2022] GET /identity/v3/projects/99b85d5359534a7a913a1ca963af82a1 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:30.179696 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540bba3-5d56-4adc-bd11-f38af6963887 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:30.251917 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:30.275854 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540bba3-5d56-4adc-bd11-f38af6963887 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:30.308731 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced219c-ae94-4a2a-a871-a063381850b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:30.309304 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced219c-ae94-4a2a-a871-a063381850b5 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:30.309578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced219c-ae94-4a2a-a871-a063381850b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:30.309842 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced219c-ae94-4a2a-a871-a063381850b5 None admin] PATH_INFO: `/v3/projects/99b85d5359534a7a913a1ca963af82a1/users/0bc07d0ca9654925b8d95d4cd2f3fc11/roles/72700efc336a4b4f992b08ff9ea99f0c` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:30.311956 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced219c-ae94-4a2a-a871-a063381850b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=99b85d5359534a7a913a1ca963af82a1, user_id=0bc07d0ca9654925b8d95d4cd2f3fc11, role_id=72700efc336a4b4f992b08ff9ea99f0c)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:30.313362 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced219c-ae94-4a2a-a871-a063381850b5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:30.325567 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 571/1123] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Sep 1 16:19:30 2022] PUT /identity/v3/projects/99b85d5359534a7a913a1ca963af82a1/users/0bc07d0ca9654925b8d95d4cd2f3fc11/roles/72700efc336a4b4f992b08ff9ea99f0c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:19:32.184396 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ff70c-8370-488d-aef6-d4888d03c80a None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:32.184927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ff70c-8370-488d-aef6-d4888d03c80a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:32.184927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ff70c-8370-488d-aef6-d4888d03c80a None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:32.186308 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 553/1124] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:19:32.191613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82082b3e-3ee2-4a74-9b8e-bd99d05babbd None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:32.192024 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82082b3e-3ee2-4a74-9b8e-bd99d05babbd None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:32.192160 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82082b3e-3ee2-4a74-9b8e-bd99d05babbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:32.210167 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-82082b3e-3ee2-4a74-9b8e-bd99d05babbd None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:32.211350 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-82082b3e-3ee2-4a74-9b8e-bd99d05babbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:32.274107 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 572/1125] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:19:32.545170 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ff70c-8370-488d-aef6-d4888d03c80a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:32.643918 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ff70c-8370-488d-aef6-d4888d03c80a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:32.646217 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:32.678054 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35abd130-1f0b-4475-86eb-9748bb400308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:32.678700 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abd130-1f0b-4475-86eb-9748bb400308 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:32.678983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abd130-1f0b-4475-86eb-9748bb400308 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:32.679254 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abd130-1f0b-4475-86eb-9748bb400308 None admin] PATH_INFO: `/v3/roles/72700efc336a4b4f992b08ff9ea99f0c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:32.680620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35abd130-1f0b-4475-86eb-9748bb400308 None admin] RBAC: Authorizing `identity:get_role(role_id=72700efc336a4b4f992b08ff9ea99f0c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:32.681984 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35abd130-1f0b-4475-86eb-9748bb400308 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:32.683322 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 554/1126] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:19:32 2022] GET /identity/v3/roles/72700efc336a4b4f992b08ff9ea99f0c => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:32.689145 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82082b3e-3ee2-4a74-9b8e-bd99d05babbd None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:32.748905 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:32.786727 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82082b3e-3ee2-4a74-9b8e-bd99d05babbd None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:32.821186 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114b99e-893b-4ea3-b61e-04db92b78bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:32.821750 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114b99e-893b-4ea3-b61e-04db92b78bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:32.822029 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114b99e-893b-4ea3-b61e-04db92b78bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:32.822332 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114b99e-893b-4ea3-b61e-04db92b78bf6 None admin] PATH_INFO: `/v3/users/0bc07d0ca9654925b8d95d4cd2f3fc11` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:32.823448 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114b99e-893b-4ea3-b61e-04db92b78bf6 None admin] RBAC: Authorizing `identity:get_user(user_id=0bc07d0ca9654925b8d95d4cd2f3fc11)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:32.824862 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114b99e-893b-4ea3-b61e-04db92b78bf6 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:32.826449 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 573/1127] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:19:32 2022] GET /identity/v3/users/0bc07d0ca9654925b8d95d4cd2f3fc11 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:32.832130 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35abd130-1f0b-4475-86eb-9748bb400308 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:32.928745 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35abd130-1f0b-4475-86eb-9748bb400308 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:32.961254 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c14b2-4b68-4656-b59e-b3f1d842681f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:32.961846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c14b2-4b68-4656-b59e-b3f1d842681f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:32.962127 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c14b2-4b68-4656-b59e-b3f1d842681f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:32.962429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c14b2-4b68-4656-b59e-b3f1d842681f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:33.001089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:33.027996 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c14b2-4b68-4656-b59e-b3f1d842681f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:33.029429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c14b2-4b68-4656-b59e-b3f1d842681f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:33.125530 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 555/1128] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:19:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:19:33.131355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114b99e-893b-4ea3-b61e-04db92b78bf6 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:33.228519 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114b99e-893b-4ea3-b61e-04db92b78bf6 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:33.257659 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:33.262882 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c493c7a-b6bd-4f1a-b094-efa38f1f36e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:33.263501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c493c7a-b6bd-4f1a-b094-efa38f1f36e0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:33.263779 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c493c7a-b6bd-4f1a-b094-efa38f1f36e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:33.264054 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c493c7a-b6bd-4f1a-b094-efa38f1f36e0 None admin] PATH_INFO: `/v3/projects/99b85d5359534a7a913a1ca963af82a1` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:33.265166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c493c7a-b6bd-4f1a-b094-efa38f1f36e0 None admin] RBAC: Authorizing `identity:get_project(project_id=99b85d5359534a7a913a1ca963af82a1)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:33.266625 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c493c7a-b6bd-4f1a-b094-efa38f1f36e0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:33.268233 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 574/1129] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:19:33 2022] GET /identity/v3/projects/99b85d5359534a7a913a1ca963af82a1 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:33.274266 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c14b2-4b68-4656-b59e-b3f1d842681f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:33.341672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:33.371122 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c14b2-4b68-4656-b59e-b3f1d842681f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:33.403728 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4153be6-2541-4d7f-99e1-e0f0b5a0eb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:33.404255 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4153be6-2541-4d7f-99e1-e0f0b5a0eb36 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:33.404532 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4153be6-2541-4d7f-99e1-e0f0b5a0eb36 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:33.404800 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4153be6-2541-4d7f-99e1-e0f0b5a0eb36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:33.405510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4153be6-2541-4d7f-99e1-e0f0b5a0eb36 None admin] RBAC: Adding query filter params (role.id=72700efc336a4b4f992b08ff9ea99f0c, scope.project.id=99b85d5359534a7a913a1ca963af82a1, user.id=0bc07d0ca9654925b8d95d4cd2f3fc11) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:33.405820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4153be6-2541-4d7f-99e1-e0f0b5a0eb36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:33.407257 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4153be6-2541-4d7f-99e1-e0f0b5a0eb36 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:33.413250 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 556/1130] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:19:33 2022] GET /identity/v3/role_assignments?user.id=0bc07d0ca9654925b8d95d4cd2f3fc11&scope.project.id=99b85d5359534a7a913a1ca963af82a1&role.id=72700efc336a4b4f992b08ff9ea99f0c => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:35.313814 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fb109-10d1-4691-ad64-c1b8813c4ae9 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:35.314346 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fb109-10d1-4691-ad64-c1b8813c4ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:35.314346 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fb109-10d1-4691-ad64-c1b8813c4ae9 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:35.315644 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 575/1131] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:19:35.320861 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b97f8f-f063-4606-8db0-ad616c69b1c0 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:35.321250 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b97f8f-f063-4606-8db0-ad616c69b1c0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:35.321420 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b97f8f-f063-4606-8db0-ad616c69b1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:35.339732 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-53b97f8f-f063-4606-8db0-ad616c69b1c0 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:35.340921 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-53b97f8f-f063-4606-8db0-ad616c69b1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:35.406185 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 557/1132] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:19:35.675529 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fb109-10d1-4691-ad64-c1b8813c4ae9 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:35.775367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fb109-10d1-4691-ad64-c1b8813c4ae9 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:35.808488 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb19c88-1292-4ba9-8404-477937559a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:35.809099 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb19c88-1292-4ba9-8404-477937559a66 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:35.809378 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb19c88-1292-4ba9-8404-477937559a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:35.809651 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb19c88-1292-4ba9-8404-477937559a66 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:35.810831 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb19c88-1292-4ba9-8404-477937559a66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:35.812243 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb19c88-1292-4ba9-8404-477937559a66 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:35.813769 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 576/1133] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:19:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:35.820124 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b97f8f-f063-4606-8db0-ad616c69b1c0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:35.883427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:35.916604 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b97f8f-f063-4606-8db0-ad616c69b1c0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:35.949670 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e1249-4ac1-4e5c-aa5e-f1d896c00dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:35.950234 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e1249-4ac1-4e5c-aa5e-f1d896c00dc6 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:35.950514 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e1249-4ac1-4e5c-aa5e-f1d896c00dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:35.950787 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e1249-4ac1-4e5c-aa5e-f1d896c00dc6 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:35.951434 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e1249-4ac1-4e5c-aa5e-f1d896c00dc6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:35.952953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e1249-4ac1-4e5c-aa5e-f1d896c00dc6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:35.982246 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 558/1134] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Sep 1 16:19:35 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:19:37.885028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb045b0-4ec7-4627-b73b-dab6e361f1d8 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:37.885582 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb045b0-4ec7-4627-b73b-dab6e361f1d8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:37.885582 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb045b0-4ec7-4627-b73b-dab6e361f1d8 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:37.886934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 577/1135] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:19:37.891911 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517a7f0-94be-49b7-8769-d4c0ee8a81ed None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:37.892313 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517a7f0-94be-49b7-8769-d4c0ee8a81ed None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:37.892468 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517a7f0-94be-49b7-8769-d4c0ee8a81ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:37.910696 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d517a7f0-94be-49b7-8769-d4c0ee8a81ed None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:37.911878 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d517a7f0-94be-49b7-8769-d4c0ee8a81ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:37.978635 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 559/1136] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:19:38.250890 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb045b0-4ec7-4627-b73b-dab6e361f1d8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:38.317590 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:38.350464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb045b0-4ec7-4627-b73b-dab6e361f1d8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:38.383624 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f74ea-01ef-4536-ac98-3af90a8a3f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:38.384238 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f74ea-01ef-4536-ac98-3af90a8a3f68 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:38.384520 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f74ea-01ef-4536-ac98-3af90a8a3f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:38.384793 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f74ea-01ef-4536-ac98-3af90a8a3f68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:38.385888 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f74ea-01ef-4536-ac98-3af90a8a3f68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:38.387347 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f74ea-01ef-4536-ac98-3af90a8a3f68 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:38.388883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 578/1137] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:19:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:38.394967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517a7f0-94be-49b7-8769-d4c0ee8a81ed None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:38.442632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:38.493459 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517a7f0-94be-49b7-8769-d4c0ee8a81ed None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:38.526829 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153489f-dbce-48ec-ac93-f919d372b3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:38.527367 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153489f-dbce-48ec-ac93-f919d372b3c3 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:38.527654 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153489f-dbce-48ec-ac93-f919d372b3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:38.527940 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153489f-dbce-48ec-ac93-f919d372b3c3 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:38.528595 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153489f-dbce-48ec-ac93-f919d372b3c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:38.530019 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153489f-dbce-48ec-ac93-f919d372b3c3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:38.552081 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 560/1138] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Sep 1 16:19:38 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:19:40.467318 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a30f6-8b9f-43d1-86d2-00af5a915bb7 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:40.467841 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a30f6-8b9f-43d1-86d2-00af5a915bb7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:40.467841 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a30f6-8b9f-43d1-86d2-00af5a915bb7 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:40.469191 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 579/1139] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:19:40.474409 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef937e-41c2-4606-b2eb-249eb6798c16 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:40.474834 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef937e-41c2-4606-b2eb-249eb6798c16 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:40.474936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef937e-41c2-4606-b2eb-249eb6798c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:40.494413 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b9ef937e-41c2-4606-b2eb-249eb6798c16 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:40.495612 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b9ef937e-41c2-4606-b2eb-249eb6798c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:40.559844 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 561/1140] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:19:40.825246 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a30f6-8b9f-43d1-86d2-00af5a915bb7 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:40.924072 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a30f6-8b9f-43d1-86d2-00af5a915bb7 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:40.956504 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68a198-76ee-4f45-ba71-91f01549cf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:40.957137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68a198-76ee-4f45-ba71-91f01549cf49 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:40.957420 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68a198-76ee-4f45-ba71-91f01549cf49 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:40.957695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68a198-76ee-4f45-ba71-91f01549cf49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:40.960272 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:40.968856 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68a198-76ee-4f45-ba71-91f01549cf49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:19:40.974122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-2b68a198-76ee-4f45-ba71-91f01549cf49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:19:40.974122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:40.974122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:40.974122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:40.974122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:40.974122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:40.974122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:40.974122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:40.974122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:40.974122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:40.974122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:40.974122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:40.974122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:19:40.974122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:19:40.974122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:19:40.974122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:19:40.975484 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:19:40.975484 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:19:40.975484 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:19:40.975484 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:19:40.975484 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:40.975484 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:40.975484 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:40.975484 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:40.975484 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:40.975484 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:40.975484 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:40.975484 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:40.975484 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:40.975484 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:40.975484 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:40.975484 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:40.976698 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:40.976698 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:40.976698 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:40.976698 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:40.976698 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:19:40.976698 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:19:40.976698 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:19:40.976698 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:19:40.976698 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:19:40.976698 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:19:40.976698 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:19:40.976698 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:19:40.976698 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 580/1141] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:19:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:19:40.981167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef937e-41c2-4606-b2eb-249eb6798c16 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:41.013679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:41.078852 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef937e-41c2-4606-b2eb-249eb6798c16 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:41.112040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8840c3-e188-4c3d-bbc2-fa246af81d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:41.112644 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8840c3-e188-4c3d-bbc2-fa246af81d48 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:41.112927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8840c3-e188-4c3d-bbc2-fa246af81d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:41.113808 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8840c3-e188-4c3d-bbc2-fa246af81d48 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:41.114508 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8840c3-e188-4c3d-bbc2-fa246af81d48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:41.114820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8840c3-e188-4c3d-bbc2-fa246af81d48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:41.116179 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8840c3-e188-4c3d-bbc2-fa246af81d48 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:41.123913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 562/1142] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:19:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:41.129624 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68a198-76ee-4f45-ba71-91f01549cf49 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:41.227227 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68a198-76ee-4f45-ba71-91f01549cf49 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:41.259278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb0225-3719-477c-be56-17c5824cfd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:41.259889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb0225-3719-477c-be56-17c5824cfd89 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:41.260168 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb0225-3719-477c-be56-17c5824cfd89 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:41.260442 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb0225-3719-477c-be56-17c5824cfd89 None admin] PATH_INFO: `/v3/users/f976c08b29aa4527afd2e95e984db36f` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:41.273419 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb0225-3719-477c-be56-17c5824cfd89 None admin] RBAC: Authorizing `identity:get_user(user_id=f976c08b29aa4527afd2e95e984db36f)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:41.274871 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb0225-3719-477c-be56-17c5824cfd89 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:41.276417 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 581/1143] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:19:41 2022] GET /identity/v3/users/f976c08b29aa4527afd2e95e984db36f => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:41.282086 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8840c3-e188-4c3d-bbc2-fa246af81d48 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:41.372087 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:41.378060 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8840c3-e188-4c3d-bbc2-fa246af81d48 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:41.410653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efecf0e6-7951-451e-a989-7670f78ec729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:41.411238 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecf0e6-7951-451e-a989-7670f78ec729 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:41.411513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecf0e6-7951-451e-a989-7670f78ec729 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:41.411776 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecf0e6-7951-451e-a989-7670f78ec729 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:41.478212 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efecf0e6-7951-451e-a989-7670f78ec729 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:41.479656 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efecf0e6-7951-451e-a989-7670f78ec729 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:41.572988 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:41.580136 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 563/1144] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:19:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:19:41.586639 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb0225-3719-477c-be56-17c5824cfd89 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:41.641894 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:41.686569 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb0225-3719-477c-be56-17c5824cfd89 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:41.719395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b48c8-61b0-4cb0-a455-92f683e0fc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:41.720016 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b48c8-61b0-4cb0-a455-92f683e0fc10 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:41.720296 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b48c8-61b0-4cb0-a455-92f683e0fc10 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:41.720569 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b48c8-61b0-4cb0-a455-92f683e0fc10 None admin] PATH_INFO: `/v3/projects/f204df778e2f408db5b8e77cb03c81de` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:41.721666 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b48c8-61b0-4cb0-a455-92f683e0fc10 None admin] RBAC: Authorizing `identity:get_project(project_id=f204df778e2f408db5b8e77cb03c81de)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:41.723616 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b48c8-61b0-4cb0-a455-92f683e0fc10 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:41.725214 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 582/1145] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:19:41 2022] GET /identity/v3/projects/f204df778e2f408db5b8e77cb03c81de => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:41.731336 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efecf0e6-7951-451e-a989-7670f78ec729 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:41.827119 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efecf0e6-7951-451e-a989-7670f78ec729 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:41.867486 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0997c-715a-4a2e-9d2b-1bdd7e2e2c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:41.868267 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0997c-715a-4a2e-9d2b-1bdd7e2e2c6f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:41.868552 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0997c-715a-4a2e-9d2b-1bdd7e2e2c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:41.868828 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0997c-715a-4a2e-9d2b-1bdd7e2e2c6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:41.869561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d0997c-715a-4a2e-9d2b-1bdd7e2e2c6f None admin] RBAC: Adding query filter params (role.id=ff293acb8551463aafe6970230042b89, scope.project.id=f204df778e2f408db5b8e77cb03c81de, user.id=f976c08b29aa4527afd2e95e984db36f) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:41.869968 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d0997c-715a-4a2e-9d2b-1bdd7e2e2c6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:41.872303 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d0997c-715a-4a2e-9d2b-1bdd7e2e2c6f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:41.878691 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 564/1146] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:19:41 2022] GET /identity/v3/role_assignments?user.id=f976c08b29aa4527afd2e95e984db36f&scope.project.id=f204df778e2f408db5b8e77cb03c81de&role.id=ff293acb8551463aafe6970230042b89 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:43.833291 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596458a2-b4c3-4c5f-a536-19c9d110bdb5 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:43.833861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596458a2-b4c3-4c5f-a536-19c9d110bdb5 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:43.833861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596458a2-b4c3-4c5f-a536-19c9d110bdb5 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:43.835179 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 583/1147] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:19:43.840210 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda3a03-3816-4152-aebb-a6c35f35fd0c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:43.840630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda3a03-3816-4152-aebb-a6c35f35fd0c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:43.840769 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda3a03-3816-4152-aebb-a6c35f35fd0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:43.858449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6eda3a03-3816-4152-aebb-a6c35f35fd0c None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:43.859625 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6eda3a03-3816-4152-aebb-a6c35f35fd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:43.916299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:43.929935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 565/1148] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:19:44.192524 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596458a2-b4c3-4c5f-a536-19c9d110bdb5 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:44.291166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596458a2-b4c3-4c5f-a536-19c9d110bdb5 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:44.313412 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:44.325532 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f79ea1-fc12-45ee-af68-ca350c4fedfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:44.326144 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f79ea1-fc12-45ee-af68-ca350c4fedfb None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:44.326455 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f79ea1-fc12-45ee-af68-ca350c4fedfb None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:44.326725 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f79ea1-fc12-45ee-af68-ca350c4fedfb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:44.336539 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f79ea1-fc12-45ee-af68-ca350c4fedfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:19:44.341230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-f4f79ea1-fc12-45ee-af68-ca350c4fedfb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:19:44.341230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:44.341230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:44.341230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:44.341230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:44.341230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:44.341230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:44.341230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:44.341230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:44.341230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:44.341230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:44.341230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:44.341230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:19:44.341230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:19:44.341230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:19:44.341230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:19:44.342388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:19:44.342388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:19:44.342388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:19:44.342388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:19:44.342388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:44.342388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:44.342388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:44.342388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:44.342388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:44.342388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:44.342388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:44.342388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:44.342388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:44.342388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:44.342388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:44.342388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:44.343480 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:44.343480 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:44.343480 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:44.343480 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:44.343480 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:19:44.343480 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:19:44.343480 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:19:44.343480 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:19:44.343480 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:19:44.343480 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:19:44.343480 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:19:44.343480 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:19:44.343480 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 584/1149] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:19:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:19:44.348251 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda3a03-3816-4152-aebb-a6c35f35fd0c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:44.443164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda3a03-3816-4152-aebb-a6c35f35fd0c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:44.475094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d3240-a3a1-4330-8cf5-59d35266b22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:44.475691 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d3240-a3a1-4330-8cf5-59d35266b22b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:44.475972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d3240-a3a1-4330-8cf5-59d35266b22b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:44.476246 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d3240-a3a1-4330-8cf5-59d35266b22b None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:44.476898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d3240-a3a1-4330-8cf5-59d35266b22b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:44.477200 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d3240-a3a1-4330-8cf5-59d35266b22b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:44.478588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d3240-a3a1-4330-8cf5-59d35266b22b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:44.486353 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 566/1150] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:19:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:44.492365 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f79ea1-fc12-45ee-af68-ca350c4fedfb None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:44.593489 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f79ea1-fc12-45ee-af68-ca350c4fedfb None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:44.627044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b713a4-e945-413b-b8e3-8b4cee242a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:44.627656 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b713a4-e945-413b-b8e3-8b4cee242a43 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:44.627938 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b713a4-e945-413b-b8e3-8b4cee242a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:44.628213 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b713a4-e945-413b-b8e3-8b4cee242a43 None admin] PATH_INFO: `/v3/users/f976c08b29aa4527afd2e95e984db36f` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:44.629352 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b713a4-e945-413b-b8e3-8b4cee242a43 None admin] RBAC: Authorizing `identity:get_user(user_id=f976c08b29aa4527afd2e95e984db36f)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:44.630806 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b713a4-e945-413b-b8e3-8b4cee242a43 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:44.632360 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 585/1151] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:19:44 2022] GET /identity/v3/users/f976c08b29aa4527afd2e95e984db36f => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:44.638149 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d3240-a3a1-4330-8cf5-59d35266b22b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:44.690347 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:44.735584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d3240-a3a1-4330-8cf5-59d35266b22b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:44.768692 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22562119-1f75-4107-adfd-ca3598adb572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:44.769924 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22562119-1f75-4107-adfd-ca3598adb572 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:44.770228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22562119-1f75-4107-adfd-ca3598adb572 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:44.770509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22562119-1f75-4107-adfd-ca3598adb572 None admin] PATH_INFO: `/v3/projects/f204df778e2f408db5b8e77cb03c81de` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:44.771616 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22562119-1f75-4107-adfd-ca3598adb572 None admin] RBAC: Authorizing `identity:get_project(project_id=f204df778e2f408db5b8e77cb03c81de)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:44.773041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22562119-1f75-4107-adfd-ca3598adb572 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:44.774689 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 567/1152] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:19:44 2022] GET /identity/v3/projects/f204df778e2f408db5b8e77cb03c81de => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:44.780608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b713a4-e945-413b-b8e3-8b4cee242a43 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:44.830419 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:44.879150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b713a4-e945-413b-b8e3-8b4cee242a43 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:44.911244 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff50128-a258-4bbf-a3e7-a69db1b892fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:44.911883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff50128-a258-4bbf-a3e7-a69db1b892fd None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:44.912165 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff50128-a258-4bbf-a3e7-a69db1b892fd None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:44.912438 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff50128-a258-4bbf-a3e7-a69db1b892fd None admin] PATH_INFO: `/v3/projects/f204df778e2f408db5b8e77cb03c81de/users/f976c08b29aa4527afd2e95e984db36f/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:44.914345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff50128-a258-4bbf-a3e7-a69db1b892fd None admin] RBAC: Authorizing `identity:create_grant(project_id=f204df778e2f408db5b8e77cb03c81de, user_id=f976c08b29aa4527afd2e95e984db36f, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:44.916012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff50128-a258-4bbf-a3e7-a69db1b892fd None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:44.929792 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 586/1153] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Sep 1 16:19:44 2022] PUT /identity/v3/projects/f204df778e2f408db5b8e77cb03c81de/users/f976c08b29aa4527afd2e95e984db36f/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:19:46.820036 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a93ab-f6ca-4c23-8f39-065899aded32 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:46.820591 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a93ab-f6ca-4c23-8f39-065899aded32 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:46.820591 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a93ab-f6ca-4c23-8f39-065899aded32 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:46.821903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 568/1154] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:19:46.826967 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d1d7e-7ba1-45c6-9d7b-fb70dd7c60e0 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:46.827365 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d1d7e-7ba1-45c6-9d7b-fb70dd7c60e0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:46.827506 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d1d7e-7ba1-45c6-9d7b-fb70dd7c60e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:46.845214 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-cb8d1d7e-7ba1-45c6-9d7b-fb70dd7c60e0 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:46.846426 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-cb8d1d7e-7ba1-45c6-9d7b-fb70dd7c60e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:46.911347 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 587/1155] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:19:47.180201 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a93ab-f6ca-4c23-8f39-065899aded32 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:47.277456 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a93ab-f6ca-4c23-8f39-065899aded32 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:47.296822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:47.310496 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd2d561-f45c-4566-b3fa-d25e5140f189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:47.311105 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2d561-f45c-4566-b3fa-d25e5140f189 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:47.311383 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2d561-f45c-4566-b3fa-d25e5140f189 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:47.311656 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2d561-f45c-4566-b3fa-d25e5140f189 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:47.321760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd2d561-f45c-4566-b3fa-d25e5140f189 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:19:47.326438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-9fd2d561-f45c-4566-b3fa-d25e5140f189 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:19:47.326438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:47.326438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:47.326438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:47.326438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:47.326438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:47.326438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:47.326438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:47.326438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:47.326438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:47.326438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:47.326438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:47.326438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:19:47.326438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:19:47.326438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:19:47.326438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:19:47.327568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:19:47.327568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:19:47.327568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:19:47.327568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:19:47.327568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:47.327568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:47.327568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:47.327568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:47.327568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:47.327568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:47.327568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:47.327568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:47.327568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:47.327568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:47.327568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:47.327568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:47.328661 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:47.328661 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:47.328661 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:47.328661 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:47.328661 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:19:47.328661 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:19:47.328661 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:19:47.328661 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:19:47.328661 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:19:47.328661 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:19:47.328661 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:19:47.328661 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:19:47.328661 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 569/1156] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:19:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:19:47.334004 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d1d7e-7ba1-45c6-9d7b-fb70dd7c60e0 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:47.373785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:47.429845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d1d7e-7ba1-45c6-9d7b-fb70dd7c60e0 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:47.462122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb5c8a-f0cc-4ac4-880a-41c3e66d1bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:47.462693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb5c8a-f0cc-4ac4-880a-41c3e66d1bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:47.462957 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb5c8a-f0cc-4ac4-880a-41c3e66d1bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:47.463225 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb5c8a-f0cc-4ac4-880a-41c3e66d1bb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:47.463883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb5c8a-f0cc-4ac4-880a-41c3e66d1bb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:47.464181 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb5c8a-f0cc-4ac4-880a-41c3e66d1bb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:47.465525 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb5c8a-f0cc-4ac4-880a-41c3e66d1bb0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:47.473145 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 588/1157] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:19:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:47.478941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd2d561-f45c-4566-b3fa-d25e5140f189 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:47.580518 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd2d561-f45c-4566-b3fa-d25e5140f189 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:47.612806 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18106af9-330e-447f-9c1c-b92d3fc2dc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:47.613426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18106af9-330e-447f-9c1c-b92d3fc2dc19 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:47.613709 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18106af9-330e-447f-9c1c-b92d3fc2dc19 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:47.613985 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18106af9-330e-447f-9c1c-b92d3fc2dc19 None admin] PATH_INFO: `/v3/users/f976c08b29aa4527afd2e95e984db36f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:47.615119 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18106af9-330e-447f-9c1c-b92d3fc2dc19 None admin] RBAC: Authorizing `identity:get_user(user_id=f976c08b29aa4527afd2e95e984db36f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:47.616542 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18106af9-330e-447f-9c1c-b92d3fc2dc19 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:47.618105 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 570/1158] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:19:47 2022] GET /identity/v3/users/f976c08b29aa4527afd2e95e984db36f => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:47.623996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb5c8a-f0cc-4ac4-880a-41c3e66d1bb0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:47.721355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb5c8a-f0cc-4ac4-880a-41c3e66d1bb0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:47.722803 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:47.754451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b03f9a-fb01-4554-828a-c2b21e35fab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:47.755040 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b03f9a-fb01-4554-828a-c2b21e35fab9 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:47.755319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b03f9a-fb01-4554-828a-c2b21e35fab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:47.755589 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b03f9a-fb01-4554-828a-c2b21e35fab9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:47.820710 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b03f9a-fb01-4554-828a-c2b21e35fab9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:47.822131 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b03f9a-fb01-4554-828a-c2b21e35fab9 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:47.915342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 589/1159] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:19:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:19:47.921619 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18106af9-330e-447f-9c1c-b92d3fc2dc19 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:47.969319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:48.022923 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18106af9-330e-447f-9c1c-b92d3fc2dc19 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:48.056169 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a68b04-00f4-4a7e-b306-162a28b6bf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:48.056793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a68b04-00f4-4a7e-b306-162a28b6bf1b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:48.057073 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a68b04-00f4-4a7e-b306-162a28b6bf1b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:48.057345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a68b04-00f4-4a7e-b306-162a28b6bf1b None admin] PATH_INFO: `/v3/projects/f204df778e2f408db5b8e77cb03c81de` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:48.058492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a68b04-00f4-4a7e-b306-162a28b6bf1b None admin] RBAC: Authorizing `identity:get_project(project_id=f204df778e2f408db5b8e77cb03c81de)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:48.059918 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a68b04-00f4-4a7e-b306-162a28b6bf1b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:48.061545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 571/1160] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:19:47 2022] GET /identity/v3/projects/f204df778e2f408db5b8e77cb03c81de => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:48.067622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b03f9a-fb01-4554-828a-c2b21e35fab9 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:48.069778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:48.165402 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b03f9a-fb01-4554-828a-c2b21e35fab9 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:48.198374 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032adf5-f36f-4e3c-9300-27168fa4acf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:48.198968 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032adf5-f36f-4e3c-9300-27168fa4acf4 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:48.199247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032adf5-f36f-4e3c-9300-27168fa4acf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:48.199519 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032adf5-f36f-4e3c-9300-27168fa4acf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:48.200318 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d032adf5-f36f-4e3c-9300-27168fa4acf4 None admin] RBAC: Adding query filter params (role.id=ff293acb8551463aafe6970230042b89, scope.project.id=f204df778e2f408db5b8e77cb03c81de, user.id=f976c08b29aa4527afd2e95e984db36f) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:48.200626 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d032adf5-f36f-4e3c-9300-27168fa4acf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:48.202042 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d032adf5-f36f-4e3c-9300-27168fa4acf4 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:48.208157 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 590/1161] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:19:48 2022] GET /identity/v3/role_assignments?user.id=f976c08b29aa4527afd2e95e984db36f&scope.project.id=f204df778e2f408db5b8e77cb03c81de&role.id=ff293acb8551463aafe6970230042b89 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:50.135594 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f536d1-b8b4-4439-be8b-6a6072525bb8 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:50.136170 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f536d1-b8b4-4439-be8b-6a6072525bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:50.136170 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f536d1-b8b4-4439-be8b-6a6072525bb8 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:50.137488 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 572/1162] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:19:50.142674 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e21889-ee84-4938-9294-4fb7383a336c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:50.143059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e21889-ee84-4938-9294-4fb7383a336c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:50.143226 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e21889-ee84-4938-9294-4fb7383a336c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:50.161971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-29e21889-ee84-4938-9294-4fb7383a336c None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:50.163187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-29e21889-ee84-4938-9294-4fb7383a336c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:50.219763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:50.227712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 591/1163] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:19:50.495276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f536d1-b8b4-4439-be8b-6a6072525bb8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:50.594002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f536d1-b8b4-4439-be8b-6a6072525bb8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:50.609628 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:50.627446 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75361bef-bef8-4b6a-afa4-20c3267871eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:50.628061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75361bef-bef8-4b6a-afa4-20c3267871eb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:50.628343 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75361bef-bef8-4b6a-afa4-20c3267871eb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:50.628616 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75361bef-bef8-4b6a-afa4-20c3267871eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:50.693361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75361bef-bef8-4b6a-afa4-20c3267871eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:50.694847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75361bef-bef8-4b6a-afa4-20c3267871eb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:50.790898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 573/1164] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:19:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 16:19:50.796617 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e21889-ee84-4938-9294-4fb7383a336c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:50.892105 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e21889-ee84-4938-9294-4fb7383a336c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:50.924116 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d5ff2-f48c-4d8c-857f-fac03f733b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:50.925207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d5ff2-f48c-4d8c-857f-fac03f733b62 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:50.925483 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d5ff2-f48c-4d8c-857f-fac03f733b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:50.925748 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d5ff2-f48c-4d8c-857f-fac03f733b62 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:50.932098 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d5ff2-f48c-4d8c-857f-fac03f733b62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:50.933508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d5ff2-f48c-4d8c-857f-fac03f733b62 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:50.944996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-046d5ff2-f48c-4d8c-857f-fac03f733b62 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 16:19:50.944996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:50.944996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 01 16:19:50.944996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 01 16:19:50.944996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:19:50.944996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:19:50.944996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:19:50.944996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:19:50.944996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 01 16:19:50.944996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:19:50.944996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:19:50.944996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:19:50.944996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:19:50.944996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:19:50.944996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:19:50.944996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:19:50.944996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:19:50.946256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:19:50.946256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:19:50.946256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:19:50.946256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:19:50.946256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:19:50.946256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:19:50.946256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:19:50.946256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:19:50.946256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:19:50.946256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:19:50.946256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:19:50.946256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:19:50.946256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:19:50.946256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:19:50.946256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:19:50.946256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:19:50.947403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:19:50.947403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:19:50.947403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:19:50.947403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:19:50.947403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:19:50.947403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:19:50.947403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:19:50.947403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:19:50.947403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:19:50.947403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:19:50.947403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 01 16:19:50.947403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:19:50.947403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 16:19:50.947403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:19:50.947403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 592/1165] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Sep 1 16:19:50 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:19:50.953297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75361bef-bef8-4b6a-afa4-20c3267871eb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:50.973620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:51.054327 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75361bef-bef8-4b6a-afa4-20c3267871eb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:51.087239 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf229c4-d603-402e-bb6f-b1b153bc37d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:51.087836 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf229c4-d603-402e-bb6f-b1b153bc37d5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:51.088117 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf229c4-d603-402e-bb6f-b1b153bc37d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:51.088389 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf229c4-d603-402e-bb6f-b1b153bc37d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:51.089034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf229c4-d603-402e-bb6f-b1b153bc37d5 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:19:51.089337 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf229c4-d603-402e-bb6f-b1b153bc37d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:51.090735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf229c4-d603-402e-bb6f-b1b153bc37d5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:51.097605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 574/1166] 10.222.0.58 () {56 vars in 1115 bytes} [Thu Sep 1 16:19:50 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:52.981325 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2515f-7036-47e7-b3c5-6ab1de9e78a4 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:52.981889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2515f-7036-47e7-b3c5-6ab1de9e78a4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:52.981889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2515f-7036-47e7-b3c5-6ab1de9e78a4 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:52.983222 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 593/1167] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:19:52.988526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f85cf-de9c-4f47-ad4a-ca18499f08e4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:52.989015 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f85cf-de9c-4f47-ad4a-ca18499f08e4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:52.989501 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f85cf-de9c-4f47-ad4a-ca18499f08e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:53.010687 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-067f85cf-de9c-4f47-ad4a-ca18499f08e4 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:53.011924 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-067f85cf-de9c-4f47-ad4a-ca18499f08e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:53.092862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 575/1168] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:19:53.360591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2515f-7036-47e7-b3c5-6ab1de9e78a4 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:53.416743 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:53.460936 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2515f-7036-47e7-b3c5-6ab1de9e78a4 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:53.493317 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eab7cc7-364a-4311-bfdd-83db1125a330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:53.493903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab7cc7-364a-4311-bfdd-83db1125a330 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:53.494181 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab7cc7-364a-4311-bfdd-83db1125a330 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:53.494470 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab7cc7-364a-4311-bfdd-83db1125a330 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:53.495121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eab7cc7-364a-4311-bfdd-83db1125a330 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:53.496468 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eab7cc7-364a-4311-bfdd-83db1125a330 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:53.511720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 594/1169] 10.222.0.58 () {60 vars in 1138 bytes} [Thu Sep 1 16:19:53 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:19:55.477007 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69ed27-f8c7-475a-91ec-10d436a09ebc None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:55.477596 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69ed27-f8c7-475a-91ec-10d436a09ebc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:55.477596 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69ed27-f8c7-475a-91ec-10d436a09ebc None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:55.478948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 576/1170] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:19:55.484256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b88f4b-ca38-4c1f-860b-d22a30dfa08d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:55.484702 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b88f4b-ca38-4c1f-860b-d22a30dfa08d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:55.484798 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b88f4b-ca38-4c1f-860b-d22a30dfa08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:55.503709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-39b88f4b-ca38-4c1f-860b-d22a30dfa08d None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:55.504896 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-39b88f4b-ca38-4c1f-860b-d22a30dfa08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:55.570928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 595/1171] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:19:55.849105 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69ed27-f8c7-475a-91ec-10d436a09ebc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:55.861351 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:55.948947 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69ed27-f8c7-475a-91ec-10d436a09ebc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:55.981395 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec92095-9078-4f93-8843-bbf7988487f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:55.982000 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec92095-9078-4f93-8843-bbf7988487f2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:55.982304 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec92095-9078-4f93-8843-bbf7988487f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:55.982576 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec92095-9078-4f93-8843-bbf7988487f2 None admin] PATH_INFO: `/v3/domains/84fd2f468e924e6799cd502a9cc7db14` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:55.992431 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec92095-9078-4f93-8843-bbf7988487f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=84fd2f468e924e6799cd502a9cc7db14)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:55.993852 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec92095-9078-4f93-8843-bbf7988487f2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:55.995446 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 577/1172] 10.222.0.58 () {56 vars in 1183 bytes} [Thu Sep 1 16:19:55 2022] GET /identity/v3/domains/84fd2f468e924e6799cd502a9cc7db14 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:56.001296 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b88f4b-ca38-4c1f-860b-d22a30dfa08d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:56.046414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:56.106976 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b88f4b-ca38-4c1f-860b-d22a30dfa08d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:56.140483 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c96bae-2751-411b-9002-bbb9fac7b030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:56.141082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c96bae-2751-411b-9002-bbb9fac7b030 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:56.141360 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c96bae-2751-411b-9002-bbb9fac7b030 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:56.141627 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c96bae-2751-411b-9002-bbb9fac7b030 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:56.142302 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c96bae-2751-411b-9002-bbb9fac7b030 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:56.144255 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c96bae-2751-411b-9002-bbb9fac7b030 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:56.170012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 596/1173] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Sep 1 16:19:56 2022] POST /identity/v3/projects => generated 361 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:19:58.145630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6b725-9a0f-4029-ba3e-ff1962132265 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:58.146168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6b725-9a0f-4029-ba3e-ff1962132265 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:58.146168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6b725-9a0f-4029-ba3e-ff1962132265 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:58.147532 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 578/1174] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:19:58.152868 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abaeff1-2ec6-478d-8476-4a5b13828bb5 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:58.153282 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abaeff1-2ec6-478d-8476-4a5b13828bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:58.153412 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abaeff1-2ec6-478d-8476-4a5b13828bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:58.171882 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2abaeff1-2ec6-478d-8476-4a5b13828bb5 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:19:58.173048 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2abaeff1-2ec6-478d-8476-4a5b13828bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:19:58.237820 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 597/1175] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:19:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:19:58.513274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6b725-9a0f-4029-ba3e-ff1962132265 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:58.582912 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:58.615146 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6b725-9a0f-4029-ba3e-ff1962132265 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:58.648710 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056422d-a684-4ebf-98e8-240778645970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:58.649328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056422d-a684-4ebf-98e8-240778645970 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:58.649612 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056422d-a684-4ebf-98e8-240778645970 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:58.649885 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056422d-a684-4ebf-98e8-240778645970 None admin] PATH_INFO: `/v3/domains/84fd2f468e924e6799cd502a9cc7db14` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:58.651024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056422d-a684-4ebf-98e8-240778645970 None admin] RBAC: Authorizing `identity:get_domain(domain_id=84fd2f468e924e6799cd502a9cc7db14)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:58.652445 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056422d-a684-4ebf-98e8-240778645970 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:58.653975 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 579/1176] 10.222.0.58 () {56 vars in 1183 bytes} [Thu Sep 1 16:19:58 2022] GET /identity/v3/domains/84fd2f468e924e6799cd502a9cc7db14 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:19:58.659782 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abaeff1-2ec6-478d-8476-4a5b13828bb5 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:19:58.686873 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:19:58.762505 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abaeff1-2ec6-478d-8476-4a5b13828bb5 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:19:58.797123 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42b3a0-8997-4036-832d-646b81cfffce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:19:58.797646 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42b3a0-8997-4036-832d-646b81cfffce None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:58.797926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42b3a0-8997-4036-832d-646b81cfffce None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:58.798225 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42b3a0-8997-4036-832d-646b81cfffce None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:58.798862 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42b3a0-8997-4036-832d-646b81cfffce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:19:58.800900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42b3a0-8997-4036-832d-646b81cfffce None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:19:58.821615 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 598/1177] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Sep 1 16:19:58 2022] POST /identity/v3/users => generated 311 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:20:00.788432 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0671a-c004-4ad9-bdaa-e37d3412fed3 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:00.789006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0671a-c004-4ad9-bdaa-e37d3412fed3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:00.789006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0671a-c004-4ad9-bdaa-e37d3412fed3 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:00.790613 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 580/1178] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:00.795751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eaaa05-84da-4a04-83ca-7cd7c5da9965 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:00.796157 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eaaa05-84da-4a04-83ca-7cd7c5da9965 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:00.796310 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eaaa05-84da-4a04-83ca-7cd7c5da9965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:00.814285 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d9eaaa05-84da-4a04-83ca-7cd7c5da9965 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:00.815493 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d9eaaa05-84da-4a04-83ca-7cd7c5da9965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:00.880299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 599/1179] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:20:01.150163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d0671a-c004-4ad9-bdaa-e37d3412fed3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:01.250285 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d0671a-c004-4ad9-bdaa-e37d3412fed3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:01.282270 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60c1d1-da70-4c80-83f1-b1ab3d0ae68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:01.282883 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60c1d1-da70-4c80-83f1-b1ab3d0ae68e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:01.283167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60c1d1-da70-4c80-83f1-b1ab3d0ae68e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:01.283983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60c1d1-da70-4c80-83f1-b1ab3d0ae68e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:01.290824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:01.296184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60c1d1-da70-4c80-83f1-b1ab3d0ae68e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:01.301688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-5d60c1d1-da70-4c80-83f1-b1ab3d0ae68e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:20:01.301688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:01.301688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:01.301688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:01.301688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:01.301688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:01.301688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:01.301688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:01.301688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:01.301688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:01.301688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:01.301688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:01.301688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:01.301688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:01.301688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:01.301688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:01.302871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:01.302871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:01.302871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:01.302871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:01.302871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:01.302871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:01.302871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:01.302871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:01.302871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:01.302871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:01.302871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:01.302871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:01.302871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:01.302871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:01.302871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:01.302871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:01.304141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:01.304141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:01.304141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:01.304141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:01.304141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:01.304141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:01.304141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:01.304141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:01.304141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:01.304141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:01.304141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:20:01.304141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:20:01.304141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 581/1180] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:20:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:01.308765 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eaaa05-84da-4a04-83ca-7cd7c5da9965 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:01.323415 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:01.409779 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eaaa05-84da-4a04-83ca-7cd7c5da9965 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:01.442656 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce02ca-14f4-4486-b6f9-c7ec5f810521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:01.443252 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce02ca-14f4-4486-b6f9-c7ec5f810521 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:01.443535 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce02ca-14f4-4486-b6f9-c7ec5f810521 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:01.443811 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce02ca-14f4-4486-b6f9-c7ec5f810521 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:01.444475 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ce02ca-14f4-4486-b6f9-c7ec5f810521 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:01.444784 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ce02ca-14f4-4486-b6f9-c7ec5f810521 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:01.446143 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ce02ca-14f4-4486-b6f9-c7ec5f810521 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:01.454033 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 600/1181] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:01.460362 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60c1d1-da70-4c80-83f1-b1ab3d0ae68e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:01.558053 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60c1d1-da70-4c80-83f1-b1ab3d0ae68e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:01.589989 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a06a0e4-e98f-4899-8dc4-89be185d7148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:01.590626 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06a0e4-e98f-4899-8dc4-89be185d7148 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:01.590910 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06a0e4-e98f-4899-8dc4-89be185d7148 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:01.591186 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06a0e4-e98f-4899-8dc4-89be185d7148 None admin] PATH_INFO: `/v3/users/4c833c3d22c44b9ba01db4983cf18370` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:01.604166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a06a0e4-e98f-4899-8dc4-89be185d7148 None admin] RBAC: Authorizing `identity:get_user(user_id=4c833c3d22c44b9ba01db4983cf18370)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:01.605603 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a06a0e4-e98f-4899-8dc4-89be185d7148 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:01.607191 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 582/1182] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:01 2022] GET /identity/v3/users/4c833c3d22c44b9ba01db4983cf18370 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:01.613100 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce02ca-14f4-4486-b6f9-c7ec5f810521 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:01.687620 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:01.709904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce02ca-14f4-4486-b6f9-c7ec5f810521 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:01.741863 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7d1f9f-c015-4054-9ca4-9c584ca2f083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:01.742475 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d1f9f-c015-4054-9ca4-9c584ca2f083 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:01.742753 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d1f9f-c015-4054-9ca4-9c584ca2f083 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:01.743022 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d1f9f-c015-4054-9ca4-9c584ca2f083 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:01.807189 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7d1f9f-c015-4054-9ca4-9c584ca2f083 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:01.808677 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7d1f9f-c015-4054-9ca4-9c584ca2f083 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:01.880461 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:01.904187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 601/1183] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:20:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:20:01.910779 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a06a0e4-e98f-4899-8dc4-89be185d7148 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:01.965979 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:02.008378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a06a0e4-e98f-4899-8dc4-89be185d7148 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:02.042597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a423329-a7c8-4268-b8cd-28f4427178b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:02.043228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a423329-a7c8-4268-b8cd-28f4427178b9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:02.043512 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a423329-a7c8-4268-b8cd-28f4427178b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:02.043792 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a423329-a7c8-4268-b8cd-28f4427178b9 None admin] PATH_INFO: `/v3/projects/0fb9f7715d9642daa32e90c44a7e9ffd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:02.044923 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a423329-a7c8-4268-b8cd-28f4427178b9 None admin] RBAC: Authorizing `identity:get_project(project_id=0fb9f7715d9642daa32e90c44a7e9ffd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:02.046390 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a423329-a7c8-4268-b8cd-28f4427178b9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:02.048010 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 583/1184] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:01 2022] GET /identity/v3/projects/0fb9f7715d9642daa32e90c44a7e9ffd => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:02.054700 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7d1f9f-c015-4054-9ca4-9c584ca2f083 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:02.153903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7d1f9f-c015-4054-9ca4-9c584ca2f083 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:02.186596 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05935bb0-36b6-4fe9-ad21-f29ee4d74fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:02.187188 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05935bb0-36b6-4fe9-ad21-f29ee4d74fbb None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:02.187466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05935bb0-36b6-4fe9-ad21-f29ee4d74fbb None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:02.187737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05935bb0-36b6-4fe9-ad21-f29ee4d74fbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:02.188456 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05935bb0-36b6-4fe9-ad21-f29ee4d74fbb None admin] RBAC: Adding query filter params (role.id=ff293acb8551463aafe6970230042b89, scope.project.id=0fb9f7715d9642daa32e90c44a7e9ffd, user.id=4c833c3d22c44b9ba01db4983cf18370) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:02.188762 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05935bb0-36b6-4fe9-ad21-f29ee4d74fbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:02.190228 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05935bb0-36b6-4fe9-ad21-f29ee4d74fbb None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:02.195344 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 602/1185] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:20:02 2022] GET /identity/v3/role_assignments?user.id=4c833c3d22c44b9ba01db4983cf18370&scope.project.id=0fb9f7715d9642daa32e90c44a7e9ffd&role.id=ff293acb8551463aafe6970230042b89 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:04.057656 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1dd52-ca2a-4ae8-8e70-1d4ca3a782bc None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:04.058106 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1dd52-ca2a-4ae8-8e70-1d4ca3a782bc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:04.058219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1dd52-ca2a-4ae8-8e70-1d4ca3a782bc None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:04.059560 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 584/1186] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:04.064768 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc8028-ccbd-41bf-bfd5-0c11512d124a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:04.065191 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc8028-ccbd-41bf-bfd5-0c11512d124a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:04.065315 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc8028-ccbd-41bf-bfd5-0c11512d124a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:04.083042 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-dccc8028-ccbd-41bf-bfd5-0c11512d124a None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:04.084223 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-dccc8028-ccbd-41bf-bfd5-0c11512d124a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:04.118183 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:04.153107 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 603/1187] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:20:04.422264 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1dd52-ca2a-4ae8-8e70-1d4ca3a782bc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:04.520109 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1dd52-ca2a-4ae8-8e70-1d4ca3a782bc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:04.542984 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:04.553317 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07862a47-55eb-4ffa-a872-910c3699f1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:04.553932 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07862a47-55eb-4ffa-a872-910c3699f1ad None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:04.554242 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07862a47-55eb-4ffa-a872-910c3699f1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:04.554521 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07862a47-55eb-4ffa-a872-910c3699f1ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:04.564203 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07862a47-55eb-4ffa-a872-910c3699f1ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:04.568983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-07862a47-55eb-4ffa-a872-910c3699f1ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:20:04.568983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:04.568983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:04.568983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:04.568983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:04.568983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:04.568983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:04.568983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:04.568983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:04.568983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:04.568983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:04.568983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:04.568983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:04.568983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:04.568983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:04.568983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:04.570104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:04.570104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:04.570104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:04.570104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:04.570104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:04.570104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:04.570104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:04.570104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:04.570104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:04.570104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:04.570104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:04.570104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:04.570104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:04.570104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:04.570104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:04.570104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:04.571444 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:04.571444 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:04.571444 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:04.571444 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:04.571444 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:04.571444 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:04.571444 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:04.571444 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:04.571444 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:04.571444 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:04.571444 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:20:04.571444 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:20:04.571444 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 585/1188] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:20:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:04.576047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc8028-ccbd-41bf-bfd5-0c11512d124a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:04.670770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc8028-ccbd-41bf-bfd5-0c11512d124a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:04.702493 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff62e2-74e7-4c39-981d-e05ccca79460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:04.703018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff62e2-74e7-4c39-981d-e05ccca79460 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:04.703292 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff62e2-74e7-4c39-981d-e05ccca79460 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:04.703560 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff62e2-74e7-4c39-981d-e05ccca79460 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:04.704211 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff62e2-74e7-4c39-981d-e05ccca79460 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:04.704506 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff62e2-74e7-4c39-981d-e05ccca79460 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:04.706393 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff62e2-74e7-4c39-981d-e05ccca79460 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:04.713972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 604/1189] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:04.720164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07862a47-55eb-4ffa-a872-910c3699f1ad None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:04.816133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07862a47-55eb-4ffa-a872-910c3699f1ad None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:04.849674 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c7a73-210e-4100-a644-c85884ddde83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:04.850318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c7a73-210e-4100-a644-c85884ddde83 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:04.850597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c7a73-210e-4100-a644-c85884ddde83 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:04.850869 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c7a73-210e-4100-a644-c85884ddde83 None admin] PATH_INFO: `/v3/users/4c833c3d22c44b9ba01db4983cf18370` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:04.851974 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c7a73-210e-4100-a644-c85884ddde83 None admin] RBAC: Authorizing `identity:get_user(user_id=4c833c3d22c44b9ba01db4983cf18370)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:04.853389 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c7a73-210e-4100-a644-c85884ddde83 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:04.854981 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 586/1190] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:04 2022] GET /identity/v3/users/4c833c3d22c44b9ba01db4983cf18370 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:04.860994 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff62e2-74e7-4c39-981d-e05ccca79460 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:04.892101 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:04.959242 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff62e2-74e7-4c39-981d-e05ccca79460 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:04.992029 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ca103e-06bd-41b3-a508-103eb62da1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:04.992648 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca103e-06bd-41b3-a508-103eb62da1c9 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:04.992930 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca103e-06bd-41b3-a508-103eb62da1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:04.993202 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca103e-06bd-41b3-a508-103eb62da1c9 None admin] PATH_INFO: `/v3/projects/0fb9f7715d9642daa32e90c44a7e9ffd` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:04.994402 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ca103e-06bd-41b3-a508-103eb62da1c9 None admin] RBAC: Authorizing `identity:get_project(project_id=0fb9f7715d9642daa32e90c44a7e9ffd)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:04.995822 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ca103e-06bd-41b3-a508-103eb62da1c9 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:04.997426 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 605/1191] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:04 2022] GET /identity/v3/projects/0fb9f7715d9642daa32e90c44a7e9ffd => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:05.003437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c7a73-210e-4100-a644-c85884ddde83 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:05.055861 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:05.103206 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c7a73-210e-4100-a644-c85884ddde83 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:05.136701 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02450f4d-b2f1-40e2-8503-38149ce472e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:05.137340 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02450f4d-b2f1-40e2-8503-38149ce472e7 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:05.137619 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02450f4d-b2f1-40e2-8503-38149ce472e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:05.137891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02450f4d-b2f1-40e2-8503-38149ce472e7 None admin] PATH_INFO: `/v3/projects/0fb9f7715d9642daa32e90c44a7e9ffd/users/4c833c3d22c44b9ba01db4983cf18370/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:05.139764 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02450f4d-b2f1-40e2-8503-38149ce472e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=0fb9f7715d9642daa32e90c44a7e9ffd, user_id=4c833c3d22c44b9ba01db4983cf18370, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:05.141181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02450f4d-b2f1-40e2-8503-38149ce472e7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:05.153719 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 587/1192] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Sep 1 16:20:05 2022] PUT /identity/v3/projects/0fb9f7715d9642daa32e90c44a7e9ffd/users/4c833c3d22c44b9ba01db4983cf18370/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:20:07.035271 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7905ea-4042-4048-8ec7-a469e4649a97 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:07.035803 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7905ea-4042-4048-8ec7-a469e4649a97 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:07.035803 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7905ea-4042-4048-8ec7-a469e4649a97 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:07.037136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 606/1193] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:07.042626 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421916ba-06d6-4b02-aece-619c75114f09 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:07.043008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421916ba-06d6-4b02-aece-619c75114f09 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:07.043178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421916ba-06d6-4b02-aece-619c75114f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:07.061588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-421916ba-06d6-4b02-aece-619c75114f09 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:07.062814 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-421916ba-06d6-4b02-aece-619c75114f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:07.128114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 588/1194] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:20:07.389824 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7905ea-4042-4048-8ec7-a469e4649a97 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:07.485642 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:07.489899 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7905ea-4042-4048-8ec7-a469e4649a97 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:07.522543 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9258388b-1bef-4e72-898e-48e973264eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:07.523153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9258388b-1bef-4e72-898e-48e973264eac None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:07.523432 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9258388b-1bef-4e72-898e-48e973264eac None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:07.523705 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9258388b-1bef-4e72-898e-48e973264eac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:07.534640 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9258388b-1bef-4e72-898e-48e973264eac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:07.539342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-9258388b-1bef-4e72-898e-48e973264eac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:20:07.539342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:07.539342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:07.539342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:07.539342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:07.539342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:07.539342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:07.539342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:07.539342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:07.539342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:07.539342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:07.539342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:07.539342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:07.539342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:07.539342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:07.539342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:07.540726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:07.540726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:07.540726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:07.540726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:07.540726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:07.540726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:07.540726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:07.540726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:07.540726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:07.540726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:07.540726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:07.540726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:07.540726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:07.540726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:07.540726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:07.540726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:07.541680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:07.541680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:07.541680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:07.541680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:07.541680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:07.541680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:07.541680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:07.541680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:07.541680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:07.541680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:07.541680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:20:07.541680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:20:07.541680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 607/1195] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:20:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:07.546900 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421916ba-06d6-4b02-aece-619c75114f09 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:07.587455 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:07.641261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421916ba-06d6-4b02-aece-619c75114f09 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:07.674321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b3a011-1f1e-4e6b-8d83-1f0a641fc758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:07.674914 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3a011-1f1e-4e6b-8d83-1f0a641fc758 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:07.675192 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3a011-1f1e-4e6b-8d83-1f0a641fc758 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:07.675461 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3a011-1f1e-4e6b-8d83-1f0a641fc758 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:07.676115 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b3a011-1f1e-4e6b-8d83-1f0a641fc758 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:07.676413 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b3a011-1f1e-4e6b-8d83-1f0a641fc758 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:07.677760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b3a011-1f1e-4e6b-8d83-1f0a641fc758 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:07.685564 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 589/1196] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:07.691273 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9258388b-1bef-4e72-898e-48e973264eac None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:07.788790 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9258388b-1bef-4e72-898e-48e973264eac None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:07.821154 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e9166-f0ce-449c-aff8-2c96207772ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:07.821761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e9166-f0ce-449c-aff8-2c96207772ac None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:07.822040 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e9166-f0ce-449c-aff8-2c96207772ac None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:07.822457 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e9166-f0ce-449c-aff8-2c96207772ac None admin] PATH_INFO: `/v3/users/4c833c3d22c44b9ba01db4983cf18370` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:07.823696 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35e9166-f0ce-449c-aff8-2c96207772ac None admin] RBAC: Authorizing `identity:get_user(user_id=4c833c3d22c44b9ba01db4983cf18370)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:07.825109 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35e9166-f0ce-449c-aff8-2c96207772ac None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:07.826674 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 608/1197] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:07 2022] GET /identity/v3/users/4c833c3d22c44b9ba01db4983cf18370 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:07.833032 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b3a011-1f1e-4e6b-8d83-1f0a641fc758 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:07.930070 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b3a011-1f1e-4e6b-8d83-1f0a641fc758 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:07.932350 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:07.963480 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66c994-f914-4e36-9771-09a9b9eb95c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:07.964079 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66c994-f914-4e36-9771-09a9b9eb95c7 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:07.964370 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66c994-f914-4e36-9771-09a9b9eb95c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:07.964645 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66c994-f914-4e36-9771-09a9b9eb95c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:08.029104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f66c994-f914-4e36-9771-09a9b9eb95c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:08.030552 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f66c994-f914-4e36-9771-09a9b9eb95c7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:08.125004 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 590/1198] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:20:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:20:08.130899 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e9166-f0ce-449c-aff8-2c96207772ac None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:08.159263 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:08.232047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e9166-f0ce-449c-aff8-2c96207772ac None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:08.263304 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe26673-562d-4d7f-b0a7-6fa88daabcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:08.263910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe26673-562d-4d7f-b0a7-6fa88daabcd0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:08.264185 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe26673-562d-4d7f-b0a7-6fa88daabcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:08.264454 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe26673-562d-4d7f-b0a7-6fa88daabcd0 None admin] PATH_INFO: `/v3/projects/0fb9f7715d9642daa32e90c44a7e9ffd` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:08.265536 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe26673-562d-4d7f-b0a7-6fa88daabcd0 None admin] RBAC: Authorizing `identity:get_project(project_id=0fb9f7715d9642daa32e90c44a7e9ffd)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:08.266976 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe26673-562d-4d7f-b0a7-6fa88daabcd0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:08.269093 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 609/1199] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:08 2022] GET /identity/v3/projects/0fb9f7715d9642daa32e90c44a7e9ffd => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:08.275199 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66c994-f914-4e36-9771-09a9b9eb95c7 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:08.278245 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:08.373310 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66c994-f914-4e36-9771-09a9b9eb95c7 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:08.405481 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b8dab-3835-474d-9655-f3fcefdfd654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:08.406129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b8dab-3835-474d-9655-f3fcefdfd654 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:08.406435 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b8dab-3835-474d-9655-f3fcefdfd654 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:08.406704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b8dab-3835-474d-9655-f3fcefdfd654 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:08.407419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b8dab-3835-474d-9655-f3fcefdfd654 None admin] RBAC: Adding query filter params (role.id=ff293acb8551463aafe6970230042b89, scope.project.id=0fb9f7715d9642daa32e90c44a7e9ffd, user.id=4c833c3d22c44b9ba01db4983cf18370) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:08.407722 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b8dab-3835-474d-9655-f3fcefdfd654 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:08.409130 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b8dab-3835-474d-9655-f3fcefdfd654 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:08.414635 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 591/1200] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:20:08 2022] GET /identity/v3/role_assignments?user.id=4c833c3d22c44b9ba01db4983cf18370&scope.project.id=0fb9f7715d9642daa32e90c44a7e9ffd&role.id=ff293acb8551463aafe6970230042b89 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:18.259336 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=52 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:18.412112 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=66 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:03.980550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e64ecb-02ea-4e5f-8503-58f0fa250a5a None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:03.981127 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e64ecb-02ea-4e5f-8503-58f0fa250a5a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:03.981127 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e64ecb-02ea-4e5f-8503-58f0fa250a5a None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:03.983025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 610/1201] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:03.988458 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e52eb4-5d76-44e5-bc5e-b6e93fc8142a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:03.988989 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e52eb4-5d76-44e5-bc5e-b6e93fc8142a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:03.988989 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e52eb4-5d76-44e5-bc5e-b6e93fc8142a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:04.009066 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-26e52eb4-5d76-44e5-bc5e-b6e93fc8142a None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:04.010306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-26e52eb4-5d76-44e5-bc5e-b6e93fc8142a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:04.097930 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 592/1202] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:21:04.380541 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e64ecb-02ea-4e5f-8503-58f0fa250a5a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:04.533333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e64ecb-02ea-4e5f-8503-58f0fa250a5a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:04.576414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e4eac-8b46-4719-bcd1-030c2bd4c37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:04.577100 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e4eac-8b46-4719-bcd1-030c2bd4c37d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:04.577396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e4eac-8b46-4719-bcd1-030c2bd4c37d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:04.577668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e4eac-8b46-4719-bcd1-030c2bd4c37d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:04.594228 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e4eac-8b46-4719-bcd1-030c2bd4c37d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:04.595677 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e4eac-8b46-4719-bcd1-030c2bd4c37d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:04.610508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-b07e4eac-8b46-4719-bcd1-030c2bd4c37d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:04.610508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:04.610508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:04.610508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:04.610508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:04.610508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:04.610508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:04.610508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:04.610508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:04.610508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:04.610508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:04.610508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:04.610508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:04.610508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:04.610508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:04.610508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:04.618396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:04.618396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:04.618396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:04.618396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:04.618396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:04.618396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:04.618396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:04.618396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:04.618396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:04.618396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:04.618396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:04.618396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:04.618396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:04.618396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:04.618396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:04.618396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:04.618396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:04.619429 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:04.619429 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:04.619429 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:21:04.619429 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 611/1203] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:04.619688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e52eb4-5d76-44e5-bc5e-b6e93fc8142a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:04.774012 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e52eb4-5d76-44e5-bc5e-b6e93fc8142a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:04.812072 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:04.818915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b85f350-0778-43a1-bffd-7cbfbb17e4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:04.819986 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85f350-0778-43a1-bffd-7cbfbb17e4e3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:04.820707 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85f350-0778-43a1-bffd-7cbfbb17e4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:04.821212 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85f350-0778-43a1-bffd-7cbfbb17e4e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:04.821905 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b85f350-0778-43a1-bffd-7cbfbb17e4e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:04.822234 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b85f350-0778-43a1-bffd-7cbfbb17e4e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:04.823823 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b85f350-0778-43a1-bffd-7cbfbb17e4e3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:04.838884 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 593/1204] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:04.845146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e4eac-8b46-4719-bcd1-030c2bd4c37d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:04.911036 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:04.974677 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e4eac-8b46-4719-bcd1-030c2bd4c37d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:05.019161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e5316-b082-4a2c-8586-d8c6333bf040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:05.020675 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e5316-b082-4a2c-8586-d8c6333bf040 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:05.021173 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e5316-b082-4a2c-8586-d8c6333bf040 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:05.021703 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e5316-b082-4a2c-8586-d8c6333bf040 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:05.022630 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62e5316-b082-4a2c-8586-d8c6333bf040 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:05.024631 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62e5316-b082-4a2c-8586-d8c6333bf040 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:05.055791 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 612/1205] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Sep 1 16:21:04 2022] POST /identity/v3/users => generated 251 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:21:07.310348 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27166f-ca25-4c71-a1c3-461cba22d85b None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:07.310962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27166f-ca25-4c71-a1c3-461cba22d85b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:07.310962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27166f-ca25-4c71-a1c3-461cba22d85b None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:07.312131 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 594/1206] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:07.322226 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7257dd-756f-44eb-9e9d-3b851fe9efc0 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:07.322226 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7257dd-756f-44eb-9e9d-3b851fe9efc0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:07.322226 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7257dd-756f-44eb-9e9d-3b851fe9efc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:07.342702 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-3c7257dd-756f-44eb-9e9d-3b851fe9efc0 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:07.343809 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3c7257dd-756f-44eb-9e9d-3b851fe9efc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:07.421436 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 613/1207] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:21:07.712217 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb27166f-ca25-4c71-a1c3-461cba22d85b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:07.858666 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb27166f-ca25-4c71-a1c3-461cba22d85b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:07.907100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73743f16-9c8c-4fdc-bdda-196f249cb313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:07.907842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73743f16-9c8c-4fdc-bdda-196f249cb313 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:07.908201 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73743f16-9c8c-4fdc-bdda-196f249cb313 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:07.908551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73743f16-9c8c-4fdc-bdda-196f249cb313 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:07.926011 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73743f16-9c8c-4fdc-bdda-196f249cb313 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:07.931741 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-73743f16-9c8c-4fdc-bdda-196f249cb313 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:07.931741 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:07.931741 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:07.931741 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:07.931741 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:07.931741 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:07.931741 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:07.931741 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:07.931741 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:07.931741 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:07.931741 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:07.931741 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:07.931741 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:07.931741 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:07.931741 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:07.931741 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:07.933171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:07.933171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:07.933171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:07.933171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:07.933171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:07.933171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:07.933171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:07.933171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:07.933171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:07.933171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:07.933171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:07.933171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:07.933171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:07.933171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:07.933171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:07.933171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:07.934290 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:07.934290 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:07.934290 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:07.934290 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:07.934290 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:07.934290 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:07.934290 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:07.934290 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:07.934290 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:07.934290 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:07.934290 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:07.934290 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:21:07.937929 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 595/1208] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Sep 1 16:21:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:07.943739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7257dd-756f-44eb-9e9d-3b851fe9efc0 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:07.990663 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:08.061793 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7257dd-756f-44eb-9e9d-3b851fe9efc0 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:08.103864 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc538b-5225-4d6e-9acb-9d944f9c06f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:08.104495 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc538b-5225-4d6e-9acb-9d944f9c06f8 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:08.104775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc538b-5225-4d6e-9acb-9d944f9c06f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:08.105049 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc538b-5225-4d6e-9acb-9d944f9c06f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:08.105777 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc538b-5225-4d6e-9acb-9d944f9c06f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:08.106085 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc538b-5225-4d6e-9acb-9d944f9c06f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:08.108127 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc538b-5225-4d6e-9acb-9d944f9c06f8 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:08.120628 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 614/1209] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Sep 1 16:21:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:08.127619 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73743f16-9c8c-4fdc-bdda-196f249cb313 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:08.187905 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:08.249192 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73743f16-9c8c-4fdc-bdda-196f249cb313 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:08.290252 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c003024-bb8b-42dd-a2ba-5edec0388290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:08.290950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c003024-bb8b-42dd-a2ba-5edec0388290 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:08.291278 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c003024-bb8b-42dd-a2ba-5edec0388290 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:08.291598 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c003024-bb8b-42dd-a2ba-5edec0388290 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:08.303910 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c003024-bb8b-42dd-a2ba-5edec0388290 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:08.305610 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c003024-bb8b-42dd-a2ba-5edec0388290 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:08.323663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-2c003024-bb8b-42dd-a2ba-5edec0388290 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:08.323663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:08.323663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:08.323663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:08.323663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:08.323663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:08.323663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:08.323663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:08.323663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:08.323663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:08.323663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:08.323663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:08.323663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:08.323663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:08.323663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:08.323663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:08.331072 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:08.331072 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:08.331072 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:08.331072 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:08.331072 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:08.331072 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:08.331072 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:08.331072 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:08.331072 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:08.331072 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:08.331072 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:08.331072 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:08.331072 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:08.331072 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:08.331072 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:08.331072 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:08.331072 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:08.332536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:08.332536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:08.332536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:21:08.332536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 596/1210] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:08.334321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc538b-5225-4d6e-9acb-9d944f9c06f8 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:08.452007 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:08.456039 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc538b-5225-4d6e-9acb-9d944f9c06f8 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:08.494840 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7192b-4cf5-491b-a707-0718f3272eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:08.495467 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7192b-4cf5-491b-a707-0718f3272eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:08.495764 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7192b-4cf5-491b-a707-0718f3272eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:08.496049 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7192b-4cf5-491b-a707-0718f3272eb2 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:08.496772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c7192b-4cf5-491b-a707-0718f3272eb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:08.497629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c7192b-4cf5-491b-a707-0718f3272eb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:08.499289 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c7192b-4cf5-491b-a707-0718f3272eb2 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:08.512402 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 615/1211] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:08.519084 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c003024-bb8b-42dd-a2ba-5edec0388290 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:08.635809 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c003024-bb8b-42dd-a2ba-5edec0388290 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:08.637717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:08.675470 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e2ae8-a560-46c0-9fd9-c6255611532e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:08.676148 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e2ae8-a560-46c0-9fd9-c6255611532e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:08.676437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e2ae8-a560-46c0-9fd9-c6255611532e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:08.676726 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e2ae8-a560-46c0-9fd9-c6255611532e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:08.685960 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e2ae8-a560-46c0-9fd9-c6255611532e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:08.687516 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e2ae8-a560-46c0-9fd9-c6255611532e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:08.704771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-f52e2ae8-a560-46c0-9fd9-c6255611532e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:21:08.704771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:08.704771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:08.704771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:08.704771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:08.704771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:08.704771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:08.704771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:08.704771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:08.704771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:08.704771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:08.704771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:08.704771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:08.704771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:08.704771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:08.704771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:08.705919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:08.705919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:08.705919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:08.705919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:08.705919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:08.705919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:08.705919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:08.705919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:08.705919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:08.705919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:08.705919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:08.705919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:08.705919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:08.705919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:08.705919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:08.705919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:08.705919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:08.705919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:08.706981 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:08.706981 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:08.706981 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:08.706981 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:08.706981 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:08.706981 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:08.706981 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:08.706981 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:08.706981 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:21:08.706981 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:21:08.706981 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 597/1212] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:08.714097 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7192b-4cf5-491b-a707-0718f3272eb2 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:08.832475 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7192b-4cf5-491b-a707-0718f3272eb2 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:08.872027 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eaa174-4864-440f-966a-cd12478f772f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:08.872645 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaa174-4864-440f-966a-cd12478f772f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:08.872929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaa174-4864-440f-966a-cd12478f772f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:08.873194 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaa174-4864-440f-966a-cd12478f772f None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:08.873939 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eaa174-4864-440f-966a-cd12478f772f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:08.874394 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eaa174-4864-440f-966a-cd12478f772f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:08.876062 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eaa174-4864-440f-966a-cd12478f772f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:08.895306 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 616/1213] 10.222.0.58 () {56 vars in 1143 bytes} [Thu Sep 1 16:21:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:08.901558 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e2ae8-a560-46c0-9fd9-c6255611532e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:09.011307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e2ae8-a560-46c0-9fd9-c6255611532e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:09.053345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a791739-d5e6-4fa7-a7c2-e74c79bd246a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:09.054499 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a791739-d5e6-4fa7-a7c2-e74c79bd246a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:09.055041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a791739-d5e6-4fa7-a7c2-e74c79bd246a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:09.055526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a791739-d5e6-4fa7-a7c2-e74c79bd246a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:09.101222 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:09.144428 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a791739-d5e6-4fa7-a7c2-e74c79bd246a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:09.146020 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a791739-d5e6-4fa7-a7c2-e74c79bd246a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:09.277466 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 598/1214] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:21:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 16:21:09.283264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eaa174-4864-440f-966a-cd12478f772f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:09.304434 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:09.407535 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eaa174-4864-440f-966a-cd12478f772f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:09.445392 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e228a266-800a-4934-9d4b-94b661edd5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:09.446025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228a266-800a-4934-9d4b-94b661edd5bf None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:09.446335 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228a266-800a-4934-9d4b-94b661edd5bf None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:09.446610 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228a266-800a-4934-9d4b-94b661edd5bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:09.459362 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e228a266-800a-4934-9d4b-94b661edd5bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:09.461186 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e228a266-800a-4934-9d4b-94b661edd5bf None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:09.477594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-e228a266-800a-4934-9d4b-94b661edd5bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:09.477594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:09.477594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:09.477594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:09.477594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:09.477594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:09.477594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:09.477594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:09.477594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:09.477594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:09.477594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:09.477594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:09.477594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:09.477594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:09.477594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:09.477594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:09.478816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:09.478816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:09.478816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:09.478816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:09.478816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:09.478816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:09.478816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:09.478816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:09.478816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:09.478816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:09.478816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:09.478816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:09.478816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:09.478816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:09.478816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:09.478816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:09.478816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:09.479845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:09.479845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:09.479845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:21:09.479845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 617/1215] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:09.484779 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a791739-d5e6-4fa7-a7c2-e74c79bd246a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:09.554481 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:09.598040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a791739-d5e6-4fa7-a7c2-e74c79bd246a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:09.637437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1b7ce-787a-4e55-9f0d-fc04bc589c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:09.638854 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1b7ce-787a-4e55-9f0d-fc04bc589c48 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:09.639307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1b7ce-787a-4e55-9f0d-fc04bc589c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:09.639590 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1b7ce-787a-4e55-9f0d-fc04bc589c48 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:09.640244 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1b7ce-787a-4e55-9f0d-fc04bc589c48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:09.640555 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1b7ce-787a-4e55-9f0d-fc04bc589c48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:09.641967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1b7ce-787a-4e55-9f0d-fc04bc589c48 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:09.655169 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 599/1216] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:21:09.661163 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e228a266-800a-4934-9d4b-94b661edd5bf None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:09.741330 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:09.778477 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e228a266-800a-4934-9d4b-94b661edd5bf None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:09.819715 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04714989-aa41-4251-ace3-649f390a31de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:09.820342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04714989-aa41-4251-ace3-649f390a31de None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:09.820652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04714989-aa41-4251-ace3-649f390a31de None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:09.820928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04714989-aa41-4251-ace3-649f390a31de None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:09.832288 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04714989-aa41-4251-ace3-649f390a31de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:09.833698 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04714989-aa41-4251-ace3-649f390a31de None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:09.846481 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-04714989-aa41-4251-ace3-649f390a31de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:09.846481 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:09.846481 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:09.846481 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:09.846481 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:09.846481 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:09.846481 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:09.846481 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:09.846481 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:09.846481 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:09.846481 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:09.846481 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:09.846481 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:09.846481 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:09.846481 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:09.847517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:09.847517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:09.847517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:09.847517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:09.847517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:09.847517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:09.847517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:09.847517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:09.847517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:09.847517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:09.847517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:09.847517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:09.847517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:09.847517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:09.847517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:09.847517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:09.847517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:09.854946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:09.854946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:09.854946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:09.854946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:09.854946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:09.854946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:09.854946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:09.854946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:21:09.854946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 618/1217] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:09.855490 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1b7ce-787a-4e55-9f0d-fc04bc589c48 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:09.967561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1b7ce-787a-4e55-9f0d-fc04bc589c48 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:09.998898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:10.013781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aff405-3e61-40cd-846f-74507cd27c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:10.014481 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aff405-3e61-40cd-846f-74507cd27c07 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:10.014848 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aff405-3e61-40cd-846f-74507cd27c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:10.015029 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aff405-3e61-40cd-846f-74507cd27c07 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:10.015798 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aff405-3e61-40cd-846f-74507cd27c07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:10.016191 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aff405-3e61-40cd-846f-74507cd27c07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:10.017650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aff405-3e61-40cd-846f-74507cd27c07 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:10.031786 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 600/1218] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:10.039421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04714989-aa41-4251-ace3-649f390a31de None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:10.174133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04714989-aa41-4251-ace3-649f390a31de None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:10.214012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:10.221853 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e980e-f672-4f0a-899d-42e0c46ad2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:10.222589 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e980e-f672-4f0a-899d-42e0c46ad2ed None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:10.222932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e980e-f672-4f0a-899d-42e0c46ad2ed None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:10.223332 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e980e-f672-4f0a-899d-42e0c46ad2ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:10.224132 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e980e-f672-4f0a-899d-42e0c46ad2ed None admin] RBAC: Adding query filter params (role.id=b0923c7a240149ae9ece5b3c1622e860, scope.project.id=479530ced580426b85d1fc07af43417f, user.id=e12302cdb2854b0684179647a6dadf6e) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:10.224503 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e980e-f672-4f0a-899d-42e0c46ad2ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:10.226076 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e980e-f672-4f0a-899d-42e0c46ad2ed None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:10.233871 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 619/1219] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:21:10 2022] GET /identity/v3/role_assignments?user.id=e12302cdb2854b0684179647a6dadf6e&scope.project.id=479530ced580426b85d1fc07af43417f&role.id=b0923c7a240149ae9ece5b3c1622e860 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:21:12.151148 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d714e-ff03-4d2f-b50e-1bb9a752d5eb None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:12.151772 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d714e-ff03-4d2f-b50e-1bb9a752d5eb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:12.151772 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d714e-ff03-4d2f-b50e-1bb9a752d5eb None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:12.153176 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 601/1220] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:12.158092 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4b39b-70b0-46b4-bd09-686a92009e58 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:12.158543 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4b39b-70b0-46b4-bd09-686a92009e58 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:12.158692 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4b39b-70b0-46b4-bd09-686a92009e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:12.178015 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-c1f4b39b-70b0-46b4-bd09-686a92009e58 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:12.179224 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c1f4b39b-70b0-46b4-bd09-686a92009e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:12.251368 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 620/1221] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:21:12.523049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d714e-ff03-4d2f-b50e-1bb9a752d5eb None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:12.642807 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d714e-ff03-4d2f-b50e-1bb9a752d5eb None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:12.682623 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac11538-2a5b-49ae-bfe7-bae4d8769f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:12.683464 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac11538-2a5b-49ae-bfe7-bae4d8769f88 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:12.683745 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac11538-2a5b-49ae-bfe7-bae4d8769f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:12.684285 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac11538-2a5b-49ae-bfe7-bae4d8769f88 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:12.695789 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac11538-2a5b-49ae-bfe7-bae4d8769f88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:12.702094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-aac11538-2a5b-49ae-bfe7-bae4d8769f88 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:12.702094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:12.702094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:12.702094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:12.702094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:12.702094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:12.702094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:12.702094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:12.702094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:12.702094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:12.702094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:12.702094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:12.702094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:12.702094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:12.702094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:12.702094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:12.703297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:12.703297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:12.703297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:12.703297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:12.703297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:12.703297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:12.703297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:12.703297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:12.703297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:12.703297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:12.703297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:12.703297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:12.703297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:12.703297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:12.703297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:12.703297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:12.704520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:12.704520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:12.704520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:12.704520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:12.704520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:12.704520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:12.704520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:12.704520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:12.704520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:12.704520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:12.704520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:12.704520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:21:12.704520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 602/1222] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Sep 1 16:21:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:12.709487 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4b39b-70b0-46b4-bd09-686a92009e58 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:12.812444 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4b39b-70b0-46b4-bd09-686a92009e58 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:12.836799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:12.849138 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02d89b-6544-49ea-94d5-35c149cb87e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:12.849857 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02d89b-6544-49ea-94d5-35c149cb87e1 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:12.850109 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02d89b-6544-49ea-94d5-35c149cb87e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:12.850409 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02d89b-6544-49ea-94d5-35c149cb87e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:12.851079 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed02d89b-6544-49ea-94d5-35c149cb87e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:12.851389 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed02d89b-6544-49ea-94d5-35c149cb87e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:12.852809 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed02d89b-6544-49ea-94d5-35c149cb87e1 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:12.861921 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 621/1223] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Sep 1 16:21:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:12.867918 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac11538-2a5b-49ae-bfe7-bae4d8769f88 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:12.916087 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:12.980670 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac11538-2a5b-49ae-bfe7-bae4d8769f88 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:13.017345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c9a9b-569d-4e95-a440-2f43f1cb6df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:13.018046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c9a9b-569d-4e95-a440-2f43f1cb6df0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:13.018388 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c9a9b-569d-4e95-a440-2f43f1cb6df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:13.018710 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c9a9b-569d-4e95-a440-2f43f1cb6df0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:13.030899 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c9a9b-569d-4e95-a440-2f43f1cb6df0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:13.032337 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c9a9b-569d-4e95-a440-2f43f1cb6df0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:13.047437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-503c9a9b-569d-4e95-a440-2f43f1cb6df0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:13.047437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:13.047437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:13.047437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:13.047437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:13.047437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:13.047437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:13.047437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:13.047437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:13.047437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:13.047437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:13.047437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:13.047437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:13.047437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:13.047437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:13.047437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:13.048749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:13.048749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:13.048749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:13.048749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:13.048749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:13.048749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:13.048749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:13.048749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:13.048749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:13.048749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:13.048749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:13.048749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:13.048749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:13.048749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:13.048749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:13.048749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:13.048749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:13.050257 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:13.050257 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:13.050257 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:21:13.050257 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 603/1224] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:13.055030 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02d89b-6544-49ea-94d5-35c149cb87e1 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:13.168108 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02d89b-6544-49ea-94d5-35c149cb87e1 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:13.204895 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d8cc98-9787-4d93-88bd-6f784ef8db29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:13.205501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8cc98-9787-4d93-88bd-6f784ef8db29 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:13.205777 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8cc98-9787-4d93-88bd-6f784ef8db29 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:13.206051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8cc98-9787-4d93-88bd-6f784ef8db29 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:13.206719 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d8cc98-9787-4d93-88bd-6f784ef8db29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:13.207018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d8cc98-9787-4d93-88bd-6f784ef8db29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:13.208379 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d8cc98-9787-4d93-88bd-6f784ef8db29 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:13.226264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 622/1225] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:13.227640 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c9a9b-569d-4e95-a440-2f43f1cb6df0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:13.331962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:13.339466 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c9a9b-569d-4e95-a440-2f43f1cb6df0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:13.377912 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be0a3c-65af-4a55-98d7-175cd6655ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:13.378666 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be0a3c-65af-4a55-98d7-175cd6655ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:13.378943 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be0a3c-65af-4a55-98d7-175cd6655ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:13.379218 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be0a3c-65af-4a55-98d7-175cd6655ad6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:13.387320 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44be0a3c-65af-4a55-98d7-175cd6655ad6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:13.388791 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44be0a3c-65af-4a55-98d7-175cd6655ad6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:13.400899 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-44be0a3c-65af-4a55-98d7-175cd6655ad6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:21:13.400899 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:13.400899 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:13.400899 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:13.400899 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:13.400899 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:13.400899 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:13.400899 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:13.400899 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:13.400899 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:13.400899 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:13.400899 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:13.400899 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:13.400899 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:13.400899 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:13.400899 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:13.402061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:13.402061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:13.402061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:13.402061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:13.402061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:13.402061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:13.402061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:13.402061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:13.402061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:13.402061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:13.402061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:13.402061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:13.402061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:13.402061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:13.402061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:13.402061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:13.402061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:13.402061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:13.403418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:13.403418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:13.403418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:13.403418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:13.403418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:13.403418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:13.403418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:13.403418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:13.403418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:21:13.403418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:21:13.403418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 604/1226] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:13.408207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d8cc98-9787-4d93-88bd-6f784ef8db29 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:13.462741 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:13.523860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d8cc98-9787-4d93-88bd-6f784ef8db29 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:13.571125 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd123fdf-bd20-43f2-ac41-455de6902761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:13.571722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd123fdf-bd20-43f2-ac41-455de6902761 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:13.571999 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd123fdf-bd20-43f2-ac41-455de6902761 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:13.572266 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd123fdf-bd20-43f2-ac41-455de6902761 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:13.573004 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd123fdf-bd20-43f2-ac41-455de6902761 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:13.573304 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd123fdf-bd20-43f2-ac41-455de6902761 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:13.575058 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd123fdf-bd20-43f2-ac41-455de6902761 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:13.594500 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 623/1227] 10.222.0.58 () {56 vars in 1143 bytes} [Thu Sep 1 16:21:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:13.600971 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be0a3c-65af-4a55-98d7-175cd6655ad6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:13.712859 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be0a3c-65af-4a55-98d7-175cd6655ad6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:13.750024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc4ad5-5632-4f2b-b4b7-ea8ce2682f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:13.750739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc4ad5-5632-4f2b-b4b7-ea8ce2682f82 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:13.751079 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc4ad5-5632-4f2b-b4b7-ea8ce2682f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:13.751352 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc4ad5-5632-4f2b-b4b7-ea8ce2682f82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:13.765648 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc4ad5-5632-4f2b-b4b7-ea8ce2682f82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:13.767277 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc4ad5-5632-4f2b-b4b7-ea8ce2682f82 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:13.781584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-92cc4ad5-5632-4f2b-b4b7-ea8ce2682f82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:13.781584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:13.781584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:13.781584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:13.781584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:13.781584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:13.781584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:13.781584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:13.781584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:13.781584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:13.781584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:13.781584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:13.781584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:13.781584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:13.781584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:13.781584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:13.782845 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:13.782845 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:13.782845 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:13.782845 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:13.782845 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:13.782845 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:13.782845 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:13.782845 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:13.782845 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:13.782845 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:13.782845 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:13.782845 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:13.782845 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:13.782845 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:13.782845 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:13.782845 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:13.782845 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:13.783986 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:13.783986 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:13.783986 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:21:13.786589 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 605/1228] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:13.789185 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd123fdf-bd20-43f2-ac41-455de6902761 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:13.896782 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd123fdf-bd20-43f2-ac41-455de6902761 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:13.899511 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:13.935533 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d631a7-eb45-4556-9cac-e3873b2602cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:13.936301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d631a7-eb45-4556-9cac-e3873b2602cc None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:13.936623 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d631a7-eb45-4556-9cac-e3873b2602cc None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:13.936919 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d631a7-eb45-4556-9cac-e3873b2602cc None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:13.937649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d631a7-eb45-4556-9cac-e3873b2602cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:13.937974 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d631a7-eb45-4556-9cac-e3873b2602cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:13.939528 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d631a7-eb45-4556-9cac-e3873b2602cc None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:13.952897 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 624/1229] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:13.960004 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc4ad5-5632-4f2b-b4b7-ea8ce2682f82 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:13.970502 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:14.072680 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc4ad5-5632-4f2b-b4b7-ea8ce2682f82 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:14.115737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59cf89-0b6b-4057-a236-871e9832bc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:14.116371 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59cf89-0b6b-4057-a236-871e9832bc8e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:14.116647 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59cf89-0b6b-4057-a236-871e9832bc8e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:14.116929 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59cf89-0b6b-4057-a236-871e9832bc8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:14.125496 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e59cf89-0b6b-4057-a236-871e9832bc8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:14.127396 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e59cf89-0b6b-4057-a236-871e9832bc8e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:14.141253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-3e59cf89-0b6b-4057-a236-871e9832bc8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:14.141253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:14.141253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:14.141253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:14.141253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:14.141253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:14.141253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:14.141253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:14.141253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:14.141253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:14.141253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:14.141253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:14.141253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:14.141253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:14.141253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:14.142440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:14.142440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:14.142440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:14.142440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:14.142440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:14.142440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:14.142440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:14.142440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:14.142440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:14.142440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:14.142440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:14.142440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:14.142440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:14.142440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:14.142440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:14.142440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:14.142440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:14.143555 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:14.143555 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:14.143555 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:14.143555 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:14.143555 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:14.143555 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:14.143555 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:14.143555 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:21:14.143555 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 606/1230] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:14.148875 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d631a7-eb45-4556-9cac-e3873b2602cc None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:14.261034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d631a7-eb45-4556-9cac-e3873b2602cc None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:14.299115 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebb9f8-f917-43b1-83d7-9b08f207d851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:14.299736 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebb9f8-f917-43b1-83d7-9b08f207d851 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:14.300012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebb9f8-f917-43b1-83d7-9b08f207d851 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:14.300288 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebb9f8-f917-43b1-83d7-9b08f207d851 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:14.300986 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ebb9f8-f917-43b1-83d7-9b08f207d851 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:14.301303 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ebb9f8-f917-43b1-83d7-9b08f207d851 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:14.302761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ebb9f8-f917-43b1-83d7-9b08f207d851 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:14.321186 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 625/1231] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:14.322470 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59cf89-0b6b-4057-a236-871e9832bc8e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:14.392973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:14.430379 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59cf89-0b6b-4057-a236-871e9832bc8e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:14.468399 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d56c3-e672-4958-a2d7-6df718b140dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:14.469127 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d56c3-e672-4958-a2d7-6df718b140dd None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:14.469429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d56c3-e672-4958-a2d7-6df718b140dd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:14.469728 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d56c3-e672-4958-a2d7-6df718b140dd None admin] PATH_INFO: `/v3/projects/479530ced580426b85d1fc07af43417f/users/e12302cdb2854b0684179647a6dadf6e/roles/b0923c7a240149ae9ece5b3c1622e860` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:14.486385 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d56c3-e672-4958-a2d7-6df718b140dd None admin] RBAC: Authorizing `identity:create_grant(project_id=479530ced580426b85d1fc07af43417f, user_id=e12302cdb2854b0684179647a6dadf6e, role_id=b0923c7a240149ae9ece5b3c1622e860)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:14.487978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d56c3-e672-4958-a2d7-6df718b140dd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:14.502322 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 607/1232] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Sep 1 16:21:14 2022] PUT /identity/v3/projects/479530ced580426b85d1fc07af43417f/users/e12302cdb2854b0684179647a6dadf6e/roles/b0923c7a240149ae9ece5b3c1622e860 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:21:16.679585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a520b-16ba-4e06-81aa-44c52224a254 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:16.680208 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a520b-16ba-4e06-81aa-44c52224a254 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:16.680208 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a520b-16ba-4e06-81aa-44c52224a254 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:16.681463 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 626/1233] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:16.690986 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20e861-690c-4968-a070-17356fdfe0fe None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:16.691454 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20e861-690c-4968-a070-17356fdfe0fe None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:16.691534 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20e861-690c-4968-a070-17356fdfe0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:16.711503 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-af20e861-690c-4968-a070-17356fdfe0fe None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:16.712875 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-af20e861-690c-4968-a070-17356fdfe0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:16.786458 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 608/1234] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:21:17.081962 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a520b-16ba-4e06-81aa-44c52224a254 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:17.111075 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:17.196958 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a520b-16ba-4e06-81aa-44c52224a254 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:17.235057 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb71a85-d77a-47f3-af00-ecfa9bed439e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:17.235699 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb71a85-d77a-47f3-af00-ecfa9bed439e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:17.235987 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb71a85-d77a-47f3-af00-ecfa9bed439e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:17.236305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb71a85-d77a-47f3-af00-ecfa9bed439e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:17.252093 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb71a85-d77a-47f3-af00-ecfa9bed439e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:17.258969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-deb71a85-d77a-47f3-af00-ecfa9bed439e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:17.258969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:17.258969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:17.258969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:17.258969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:17.258969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:17.258969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:17.258969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:17.258969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:17.258969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:17.258969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:17.258969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:17.258969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:17.258969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:17.258969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:17.258969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:17.260077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:17.260077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:17.260077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:17.260077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:17.260077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:17.260077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:17.260077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:17.260077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:17.260077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:17.260077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:17.260077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:17.260077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:17.260077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:17.260077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:17.260077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:17.260077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:17.260946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:17.260946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:17.260946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:17.260946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:17.260946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:17.260946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:17.260946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:17.260946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:17.260946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:17.260946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:17.260946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:17.260946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:21:17.260946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 627/1235] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Sep 1 16:21:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:17.267803 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20e861-690c-4968-a070-17356fdfe0fe None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:17.328492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:17.388154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20e861-690c-4968-a070-17356fdfe0fe None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:17.423983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d4cfe-3a6e-4fad-8a16-9ec782ca0418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:17.424692 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d4cfe-3a6e-4fad-8a16-9ec782ca0418 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:17.424946 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d4cfe-3a6e-4fad-8a16-9ec782ca0418 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:17.425213 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d4cfe-3a6e-4fad-8a16-9ec782ca0418 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:17.425886 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d4cfe-3a6e-4fad-8a16-9ec782ca0418 None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:17.426186 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d4cfe-3a6e-4fad-8a16-9ec782ca0418 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:17.427579 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d4cfe-3a6e-4fad-8a16-9ec782ca0418 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:17.436728 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 609/1236] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Sep 1 16:21:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:17.443133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb71a85-d77a-47f3-af00-ecfa9bed439e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:17.543158 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:17.562335 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb71a85-d77a-47f3-af00-ecfa9bed439e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:17.603688 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ddac5b-148a-4a75-b280-6d98cf98a0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:17.604450 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ddac5b-148a-4a75-b280-6d98cf98a0bd None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:17.604758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ddac5b-148a-4a75-b280-6d98cf98a0bd None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:17.605018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ddac5b-148a-4a75-b280-6d98cf98a0bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:17.624419 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ddac5b-148a-4a75-b280-6d98cf98a0bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:17.626022 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ddac5b-148a-4a75-b280-6d98cf98a0bd None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:17.644570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-e1ddac5b-148a-4a75-b280-6d98cf98a0bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:17.644570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:17.644570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:17.644570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:17.644570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:17.644570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:17.644570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:17.644570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:17.644570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:17.644570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:17.644570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:17.644570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:17.644570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:17.644570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:17.644570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:17.644570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:17.647023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:17.647023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:17.647023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:17.647023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:17.647023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:17.647023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:17.647023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:17.647023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:17.647023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:17.647023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:17.647023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:17.647023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:17.647023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:17.647023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:17.647023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:17.647023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:17.647023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:17.649009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:17.649009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:17.649009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:21:17.649009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 628/1237] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:17.652982 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d4cfe-3a6e-4fad-8a16-9ec782ca0418 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:17.763288 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d4cfe-3a6e-4fad-8a16-9ec782ca0418 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:17.773650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:17.800418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43fc8f3-6018-4964-b3ec-4eb5ea2af246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:17.801207 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43fc8f3-6018-4964-b3ec-4eb5ea2af246 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:17.801490 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43fc8f3-6018-4964-b3ec-4eb5ea2af246 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:17.801766 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43fc8f3-6018-4964-b3ec-4eb5ea2af246 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:17.802484 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43fc8f3-6018-4964-b3ec-4eb5ea2af246 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:17.802787 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43fc8f3-6018-4964-b3ec-4eb5ea2af246 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:17.804308 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43fc8f3-6018-4964-b3ec-4eb5ea2af246 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:17.819169 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 610/1238] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:17.825764 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ddac5b-148a-4a75-b280-6d98cf98a0bd None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:17.932519 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ddac5b-148a-4a75-b280-6d98cf98a0bd None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:17.967360 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e2271-7e03-4685-83c6-396e879a854c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:17.968004 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e2271-7e03-4685-83c6-396e879a854c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:17.968272 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e2271-7e03-4685-83c6-396e879a854c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:17.968574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e2271-7e03-4685-83c6-396e879a854c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:17.971463 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:17.977108 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e2271-7e03-4685-83c6-396e879a854c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:17.978817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e2271-7e03-4685-83c6-396e879a854c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:17.992592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-725e2271-7e03-4685-83c6-396e879a854c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:21:17.992592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:17.992592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:17.992592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:17.992592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:17.992592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:17.992592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:17.992592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:17.992592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:17.992592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:17.992592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:17.992592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:17.992592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:17.992592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:17.992592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:17.992592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:17.993832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:17.993832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:17.993832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:17.993832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:17.993832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:17.993832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:17.993832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:17.993832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:17.993832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:17.993832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:17.993832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:17.993832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:17.993832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:17.993832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:17.993832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:17.993832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:17.993832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:17.993832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:17.995099 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:17.995099 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:17.995099 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:17.995099 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:17.995099 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:17.995099 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:17.995099 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:17.995099 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:17.995099 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:21:17.995099 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:21:17.995099 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 629/1239] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:18.001047 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43fc8f3-6018-4964-b3ec-4eb5ea2af246 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:18.106456 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43fc8f3-6018-4964-b3ec-4eb5ea2af246 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:18.142948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd8776-c95d-4edb-bc05-51e3cc554457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:18.143560 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd8776-c95d-4edb-bc05-51e3cc554457 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:18.143842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd8776-c95d-4edb-bc05-51e3cc554457 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:18.144118 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd8776-c95d-4edb-bc05-51e3cc554457 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:18.144853 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd8776-c95d-4edb-bc05-51e3cc554457 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:18.145168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd8776-c95d-4edb-bc05-51e3cc554457 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:18.146711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd8776-c95d-4edb-bc05-51e3cc554457 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:18.165770 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 611/1240] 10.222.0.58 () {56 vars in 1143 bytes} [Thu Sep 1 16:21:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:18.172484 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e2271-7e03-4685-83c6-396e879a854c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:18.288971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e2271-7e03-4685-83c6-396e879a854c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:18.324629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe186e03-d904-4b45-ab8e-c150ec1e119f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:18.325228 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe186e03-d904-4b45-ab8e-c150ec1e119f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:18.325519 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe186e03-d904-4b45-ab8e-c150ec1e119f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:18.325779 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe186e03-d904-4b45-ab8e-c150ec1e119f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:18.395293 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:18.402285 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe186e03-d904-4b45-ab8e-c150ec1e119f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:18.404213 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe186e03-d904-4b45-ab8e-c150ec1e119f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:18.515702 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 630/1241] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:21:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:21:18.523208 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd8776-c95d-4edb-bc05-51e3cc554457 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:18.556188 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:18.636427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd8776-c95d-4edb-bc05-51e3cc554457 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:18.676420 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7698653f-39f6-4645-928d-e526ff2d47d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:18.677062 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698653f-39f6-4645-928d-e526ff2d47d6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:18.677345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698653f-39f6-4645-928d-e526ff2d47d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:18.677617 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698653f-39f6-4645-928d-e526ff2d47d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:18.688885 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7698653f-39f6-4645-928d-e526ff2d47d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:18.690460 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7698653f-39f6-4645-928d-e526ff2d47d6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:18.705422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-7698653f-39f6-4645-928d-e526ff2d47d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:18.705422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:18.705422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:18.705422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:18.705422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:18.705422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:18.705422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:18.705422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:18.705422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:18.705422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:18.705422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:18.705422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:18.705422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:18.705422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:18.705422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:18.705422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:18.706653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:18.706653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:18.706653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:18.706653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:18.706653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:18.706653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:18.706653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:18.706653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:18.706653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:18.706653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:18.706653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:18.706653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:18.706653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:18.706653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:18.706653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:18.706653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:18.706653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:18.707766 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:18.707766 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:18.707766 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:21:18.707766 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 612/1242] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:18.712606 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe186e03-d904-4b45-ab8e-c150ec1e119f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:18.817117 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:18.823876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe186e03-d904-4b45-ab8e-c150ec1e119f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:18.860844 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc4453-f0b3-4034-9b48-2968df3ea37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:18.861761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc4453-f0b3-4034-9b48-2968df3ea37f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:18.861761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc4453-f0b3-4034-9b48-2968df3ea37f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:18.861947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc4453-f0b3-4034-9b48-2968df3ea37f None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:18.862619 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc4453-f0b3-4034-9b48-2968df3ea37f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:18.862916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc4453-f0b3-4034-9b48-2968df3ea37f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:18.864322 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc4453-f0b3-4034-9b48-2968df3ea37f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:18.876720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 631/1243] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:18.883095 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7698653f-39f6-4645-928d-e526ff2d47d6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:18.971039 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:18.995279 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7698653f-39f6-4645-928d-e526ff2d47d6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:19.035857 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4312b4bb-c61c-4db1-8abe-7ea2922c4e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:19.036629 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312b4bb-c61c-4db1-8abe-7ea2922c4e74 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:19.036958 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312b4bb-c61c-4db1-8abe-7ea2922c4e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:19.037587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312b4bb-c61c-4db1-8abe-7ea2922c4e74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:19.045838 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4312b4bb-c61c-4db1-8abe-7ea2922c4e74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:19.047372 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4312b4bb-c61c-4db1-8abe-7ea2922c4e74 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:19.058936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-4312b4bb-c61c-4db1-8abe-7ea2922c4e74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:19.058936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:19.058936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:19.058936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:19.058936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:19.058936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:19.058936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:19.058936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:19.058936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:19.058936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:19.058936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:19.058936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:19.058936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:19.058936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:19.058936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:19.060092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:19.060092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:19.060092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:19.060092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:19.060092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:19.060092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:19.060092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:19.060092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:19.060092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:19.060092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:19.060092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:19.060092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:19.060092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:19.060092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:19.060092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:19.060092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:19.060092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:19.061164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:19.061164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:19.061164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:19.061164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:19.061164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:19.061164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:19.061164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:19.061164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:21:19.061164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 613/1244] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:21:19.066309 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc4453-f0b3-4034-9b48-2968df3ea37f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:19.177441 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc4453-f0b3-4034-9b48-2968df3ea37f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:19.213858 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f62fd1-cf18-4451-beca-24e302cc194e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:19.214711 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f62fd1-cf18-4451-beca-24e302cc194e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:19.215676 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f62fd1-cf18-4451-beca-24e302cc194e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:19.215980 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f62fd1-cf18-4451-beca-24e302cc194e None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:19.216964 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f62fd1-cf18-4451-beca-24e302cc194e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:19.217332 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f62fd1-cf18-4451-beca-24e302cc194e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:19.219029 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f62fd1-cf18-4451-beca-24e302cc194e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:19.233126 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 632/1245] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:19.239412 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4312b4bb-c61c-4db1-8abe-7ea2922c4e74 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:19.346528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4312b4bb-c61c-4db1-8abe-7ea2922c4e74 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:19.383034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89852046-9274-466f-9ff0-769500d5a136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:19.383774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89852046-9274-466f-9ff0-769500d5a136 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:19.384119 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89852046-9274-466f-9ff0-769500d5a136 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:19.384464 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89852046-9274-466f-9ff0-769500d5a136 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:19.385443 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89852046-9274-466f-9ff0-769500d5a136 None admin] RBAC: Adding query filter params (role.id=b0923c7a240149ae9ece5b3c1622e860, scope.project.id=479530ced580426b85d1fc07af43417f, user.id=e12302cdb2854b0684179647a6dadf6e) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:19.385923 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89852046-9274-466f-9ff0-769500d5a136 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:19.388381 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89852046-9274-466f-9ff0-769500d5a136 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:19.395662 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 614/1246] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:21:19 2022] GET /identity/v3/role_assignments?user.id=e12302cdb2854b0684179647a6dadf6e&scope.project.id=479530ced580426b85d1fc07af43417f&role.id=b0923c7a240149ae9ece5b3c1622e860 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:21.742657 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed958cce-95e8-4089-8e30-0c4ea8f03b42 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:21.743293 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed958cce-95e8-4089-8e30-0c4ea8f03b42 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:21.743293 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed958cce-95e8-4089-8e30-0c4ea8f03b42 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:21.744512 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 633/1247] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:21:21.750998 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd15ce-ffcf-4bca-bbd3-bf0cc42e07fb None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:21.751527 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd15ce-ffcf-4bca-bbd3-bf0cc42e07fb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:21.751797 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd15ce-ffcf-4bca-bbd3-bf0cc42e07fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:21.759612 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:21.773957 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-dcfd15ce-ffcf-4bca-bbd3-bf0cc42e07fb None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:21.775195 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dcfd15ce-ffcf-4bca-bbd3-bf0cc42e07fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:21.847610 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 615/1248] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:21:22.113288 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed958cce-95e8-4089-8e30-0c4ea8f03b42 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:22.131781 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:22.224314 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed958cce-95e8-4089-8e30-0c4ea8f03b42 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:22.259502 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf622c5-7aab-4ece-8127-4afd8c0b6ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:22.260161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf622c5-7aab-4ece-8127-4afd8c0b6ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:22.260461 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf622c5-7aab-4ece-8127-4afd8c0b6ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:22.260783 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf622c5-7aab-4ece-8127-4afd8c0b6ff6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:22.270983 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf622c5-7aab-4ece-8127-4afd8c0b6ff6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:22.275765 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-adf622c5-7aab-4ece-8127-4afd8c0b6ff6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:21:22.275765 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:22.275765 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:22.275765 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:22.275765 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:22.275765 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:22.275765 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:22.275765 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:22.275765 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:22.275765 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:22.275765 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:22.275765 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:22.275765 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:22.275765 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:22.275765 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:22.275765 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:22.276951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:22.276951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:22.276951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:22.276951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:22.276951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:22.276951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:22.276951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:22.276951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:22.276951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:22.276951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:22.276951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:22.276951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:22.276951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:22.276951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:22.276951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:22.276951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:22.282708 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:22.282708 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:22.282708 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:22.282708 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:22.282708 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:22.282708 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:22.282708 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:22.282708 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:22.282708 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:22.282708 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:22.282708 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:21:22.282708 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:21:22.282708 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 634/1249] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:21:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:22.284119 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd15ce-ffcf-4bca-bbd3-bf0cc42e07fb None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:22.406679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd15ce-ffcf-4bca-bbd3-bf0cc42e07fb None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:22.439984 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:22.446731 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31c66d-3125-4e26-bb3d-999f261abd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:22.447556 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31c66d-3125-4e26-bb3d-999f261abd43 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:22.447887 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31c66d-3125-4e26-bb3d-999f261abd43 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:22.448266 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31c66d-3125-4e26-bb3d-999f261abd43 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:22.449681 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31c66d-3125-4e26-bb3d-999f261abd43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:22.450002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31c66d-3125-4e26-bb3d-999f261abd43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:22.451836 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31c66d-3125-4e26-bb3d-999f261abd43 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:22.462405 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 616/1250] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:21:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:22.468693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf622c5-7aab-4ece-8127-4afd8c0b6ff6 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:22.559622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:22.587140 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf622c5-7aab-4ece-8127-4afd8c0b6ff6 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:22.630009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d874ca-30ac-4f4e-bdee-e71f9b4e616d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:22.630717 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d874ca-30ac-4f4e-bdee-e71f9b4e616d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:22.630960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d874ca-30ac-4f4e-bdee-e71f9b4e616d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:22.631213 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d874ca-30ac-4f4e-bdee-e71f9b4e616d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:22.643150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d874ca-30ac-4f4e-bdee-e71f9b4e616d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:22.644617 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d874ca-30ac-4f4e-bdee-e71f9b4e616d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:22.663047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-82d874ca-30ac-4f4e-bdee-e71f9b4e616d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:22.663047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:22.663047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:22.663047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:22.663047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:22.663047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:22.663047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:22.663047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:22.663047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:22.663047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:22.663047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:22.663047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:22.663047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:22.663047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:22.663047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:22.663047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:22.664154 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:22.664154 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:22.664154 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:22.664154 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:22.664154 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:22.664154 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:22.664154 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:22.664154 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:22.664154 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:22.664154 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:22.664154 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:22.664154 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:22.664154 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:22.664154 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:22.664154 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:22.664154 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:22.664154 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:22.665409 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:22.665409 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:22.665409 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:21:22.665409 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 635/1251] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:21:22.671320 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31c66d-3125-4e26-bb3d-999f261abd43 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:22.794332 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31c66d-3125-4e26-bb3d-999f261abd43 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:22.829710 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed391e2-2e1b-43a1-ab65-0aa51420b1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:22.830337 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed391e2-2e1b-43a1-ab65-0aa51420b1b8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:22.830617 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed391e2-2e1b-43a1-ab65-0aa51420b1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:22.830887 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed391e2-2e1b-43a1-ab65-0aa51420b1b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:22.831525 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed391e2-2e1b-43a1-ab65-0aa51420b1b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:22.831822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed391e2-2e1b-43a1-ab65-0aa51420b1b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:22.833180 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed391e2-2e1b-43a1-ab65-0aa51420b1b8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:22.845229 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 617/1252] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:22.851384 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d874ca-30ac-4f4e-bdee-e71f9b4e616d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:22.957328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d874ca-30ac-4f4e-bdee-e71f9b4e616d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:22.985040 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:22.998739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7a4e7-9295-4429-97a0-648ea77ba46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:22.999717 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7a4e7-9295-4429-97a0-648ea77ba46b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:23.000220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7a4e7-9295-4429-97a0-648ea77ba46b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:23.000747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7a4e7-9295-4429-97a0-648ea77ba46b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:23.009711 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7a4e7-9295-4429-97a0-648ea77ba46b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:23.011174 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7a4e7-9295-4429-97a0-648ea77ba46b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:23.022126 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-6ad7a4e7-9295-4429-97a0-648ea77ba46b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:21:23.022126 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:23.022126 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:23.022126 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:23.022126 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:23.022126 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:23.022126 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:23.022126 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:23.022126 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:23.022126 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:23.022126 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:23.022126 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:23.022126 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:23.022126 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:23.022126 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:23.022126 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:23.023220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:23.023220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:23.023220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:23.023220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:23.023220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:23.023220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:23.023220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:23.023220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:23.023220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:23.023220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:23.023220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:23.023220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:23.023220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:23.023220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:23.023220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:23.023220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:23.023220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:23.023220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:23.024221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:23.024221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:23.024221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:23.024221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:23.024221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:23.024221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:23.024221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:23.024221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:23.024221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:21:23.024221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:21:23.024221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 636/1253] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:23.030034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed391e2-2e1b-43a1-ab65-0aa51420b1b8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:23.088105 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:23.135571 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed391e2-2e1b-43a1-ab65-0aa51420b1b8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:23.170702 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d6d33-cb5e-4289-b194-0d21cd2ae2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:23.171344 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d6d33-cb5e-4289-b194-0d21cd2ae2b4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:23.171628 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d6d33-cb5e-4289-b194-0d21cd2ae2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:23.171905 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d6d33-cb5e-4289-b194-0d21cd2ae2b4 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:23.172646 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d6d33-cb5e-4289-b194-0d21cd2ae2b4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:23.172960 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d6d33-cb5e-4289-b194-0d21cd2ae2b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:23.174423 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d6d33-cb5e-4289-b194-0d21cd2ae2b4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:23.192990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 618/1254] 10.222.0.58 () {56 vars in 1143 bytes} [Thu Sep 1 16:21:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:21:23.200358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7a4e7-9295-4429-97a0-648ea77ba46b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:23.310241 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7a4e7-9295-4429-97a0-648ea77ba46b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:23.349268 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0605a99e-1bb2-458b-b597-07c245892706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:23.349970 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605a99e-1bb2-458b-b597-07c245892706 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:23.350307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605a99e-1bb2-458b-b597-07c245892706 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:23.350646 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605a99e-1bb2-458b-b597-07c245892706 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:23.412093 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:23.426026 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0605a99e-1bb2-458b-b597-07c245892706 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:23.427603 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0605a99e-1bb2-458b-b597-07c245892706 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:23.544181 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 637/1255] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:21:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:21:23.550351 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d6d33-cb5e-4289-b194-0d21cd2ae2b4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:23.664814 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d6d33-cb5e-4289-b194-0d21cd2ae2b4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:23.680939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:23.707072 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb8e41-0e63-4f53-bc99-458c2e378f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:23.707696 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb8e41-0e63-4f53-bc99-458c2e378f35 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:23.708005 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb8e41-0e63-4f53-bc99-458c2e378f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:23.708336 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb8e41-0e63-4f53-bc99-458c2e378f35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:23.720865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb8e41-0e63-4f53-bc99-458c2e378f35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:23.722914 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb8e41-0e63-4f53-bc99-458c2e378f35 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:23.740171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-91bb8e41-0e63-4f53-bc99-458c2e378f35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:23.740171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:23.740171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:23.740171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:23.740171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:23.740171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:23.740171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:23.740171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:23.740171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:23.740171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:23.740171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:23.740171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:23.740171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:23.740171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:23.740171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:23.740171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:23.742262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:23.742262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:23.742262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:23.742262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:23.742262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:23.742262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:23.742262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:23.742262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:23.742262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:23.742262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:23.742262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:23.742262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:23.742262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:23.742262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:23.742262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:23.742262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:23.742262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:23.744305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:23.744305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:23.744305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:21:23.744305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 619/1256] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:21:23.748184 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0605a99e-1bb2-458b-b597-07c245892706 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:23.846558 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:23.860791 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0605a99e-1bb2-458b-b597-07c245892706 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:23.898135 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3dc70-fb47-4e69-8327-2ecd62d2bd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:23.898925 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3dc70-fb47-4e69-8327-2ecd62d2bd24 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:23.899228 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3dc70-fb47-4e69-8327-2ecd62d2bd24 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:23.899527 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3dc70-fb47-4e69-8327-2ecd62d2bd24 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:23.900308 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3dc70-fb47-4e69-8327-2ecd62d2bd24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:23.900654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3dc70-fb47-4e69-8327-2ecd62d2bd24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:23.903084 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3dc70-fb47-4e69-8327-2ecd62d2bd24 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:23.915032 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 638/1257] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:23.921460 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb8e41-0e63-4f53-bc99-458c2e378f35 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:24.030143 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb8e41-0e63-4f53-bc99-458c2e378f35 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:24.072923 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f361e5e-b19f-45f3-ae57-e4b3e40c9501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:24.073569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f361e5e-b19f-45f3-ae57-e4b3e40c9501 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:24.073853 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f361e5e-b19f-45f3-ae57-e4b3e40c9501 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:24.074174 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f361e5e-b19f-45f3-ae57-e4b3e40c9501 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:24.081728 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f361e5e-b19f-45f3-ae57-e4b3e40c9501 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:24.083230 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f361e5e-b19f-45f3-ae57-e4b3e40c9501 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:24.095605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-5f361e5e-b19f-45f3-ae57-e4b3e40c9501 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:24.095605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:24.095605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:24.095605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:24.095605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:24.095605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:24.095605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:24.095605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:24.095605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:24.095605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:24.095605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:24.095605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:24.095605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:24.095605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:24.095605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:24.096793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:24.096793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:24.096793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:24.096793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:24.096793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:24.096793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:24.096793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:24.096793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:24.096793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:24.096793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:24.096793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:24.096793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:24.096793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:24.096793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:24.096793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:24.096793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:24.096793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:24.097776 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:24.097776 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:24.097776 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:24.097776 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:24.097776 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:24.097776 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:24.097776 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:24.097776 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:21:24.097776 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 620/1258] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:24.102899 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3dc70-fb47-4e69-8327-2ecd62d2bd24 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:24.208479 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3dc70-fb47-4e69-8327-2ecd62d2bd24 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:24.244577 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb827dfc-181c-4df4-81ef-7317bb10292c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:24.245192 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb827dfc-181c-4df4-81ef-7317bb10292c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:24.245471 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb827dfc-181c-4df4-81ef-7317bb10292c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:24.245740 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb827dfc-181c-4df4-81ef-7317bb10292c None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:24.247002 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb827dfc-181c-4df4-81ef-7317bb10292c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:24.247309 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb827dfc-181c-4df4-81ef-7317bb10292c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:24.248757 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb827dfc-181c-4df4-81ef-7317bb10292c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:24.262682 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 639/1259] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:21:24.268911 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f361e5e-b19f-45f3-ae57-e4b3e40c9501 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:24.295880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:24.373655 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f361e5e-b19f-45f3-ae57-e4b3e40c9501 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:24.414737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d349e2a9-ad1a-4cff-a8f5-0ce5d210ce44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:24.415347 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349e2a9-ad1a-4cff-a8f5-0ce5d210ce44 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:24.415627 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349e2a9-ad1a-4cff-a8f5-0ce5d210ce44 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:24.415898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349e2a9-ad1a-4cff-a8f5-0ce5d210ce44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:24.416651 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d349e2a9-ad1a-4cff-a8f5-0ce5d210ce44 None admin] RBAC: Adding query filter params (role.id=ff293acb8551463aafe6970230042b89, scope.project.id=479530ced580426b85d1fc07af43417f, user.id=e12302cdb2854b0684179647a6dadf6e) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:24.416958 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d349e2a9-ad1a-4cff-a8f5-0ce5d210ce44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:24.418496 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d349e2a9-ad1a-4cff-a8f5-0ce5d210ce44 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:24.426903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 621/1260] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:21:24 2022] GET /identity/v3/role_assignments?user.id=e12302cdb2854b0684179647a6dadf6e&scope.project.id=479530ced580426b85d1fc07af43417f&role.id=ff293acb8551463aafe6970230042b89 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:26.334422 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a1d28-4f1d-4e77-8104-24f4bc9f0fd9 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:26.334969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a1d28-4f1d-4e77-8104-24f4bc9f0fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:26.334969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a1d28-4f1d-4e77-8104-24f4bc9f0fd9 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:26.336218 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 640/1261] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:26.342015 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d71fa-c1f0-4b3e-b371-4a285ae1be51 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:26.342483 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d71fa-c1f0-4b3e-b371-4a285ae1be51 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:26.342586 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d71fa-c1f0-4b3e-b371-4a285ae1be51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:26.360376 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-719d71fa-c1f0-4b3e-b371-4a285ae1be51 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:26.361556 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-719d71fa-c1f0-4b3e-b371-4a285ae1be51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:26.425773 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 622/1262] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:21:26.694791 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a1d28-4f1d-4e77-8104-24f4bc9f0fd9 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:26.706386 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:26.793280 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a1d28-4f1d-4e77-8104-24f4bc9f0fd9 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:26.826718 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9238f32c-b3b2-4593-ae33-c75abbca157b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:26.827323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238f32c-b3b2-4593-ae33-c75abbca157b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:26.827600 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238f32c-b3b2-4593-ae33-c75abbca157b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:26.827869 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238f32c-b3b2-4593-ae33-c75abbca157b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:26.837479 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9238f32c-b3b2-4593-ae33-c75abbca157b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:26.842217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-9238f32c-b3b2-4593-ae33-c75abbca157b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:21:26.842217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:26.842217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:26.842217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:26.842217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:26.842217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:26.842217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:26.842217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:26.842217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:26.842217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:26.842217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:26.842217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:26.842217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:26.842217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:26.842217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:26.842217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:26.843466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:26.843466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:26.843466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:26.843466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:26.843466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:26.843466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:26.843466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:26.843466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:26.843466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:26.843466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:26.843466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:26.843466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:26.843466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:26.843466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:26.843466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:26.843466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:26.844501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:26.844501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:26.844501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:26.844501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:26.844501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:26.844501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:26.844501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:26.844501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:26.844501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:26.844501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:26.844501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:21:26.844501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:21:26.844501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 641/1263] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:21:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:26.849136 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d71fa-c1f0-4b3e-b371-4a285ae1be51 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:26.867523 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:26.946919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d71fa-c1f0-4b3e-b371-4a285ae1be51 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:26.979046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997eb9b6-40d5-4eee-87bd-066bb541016d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:26.979638 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997eb9b6-40d5-4eee-87bd-066bb541016d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:26.980439 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997eb9b6-40d5-4eee-87bd-066bb541016d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:26.980715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997eb9b6-40d5-4eee-87bd-066bb541016d None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:26.981377 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997eb9b6-40d5-4eee-87bd-066bb541016d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:26.981682 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997eb9b6-40d5-4eee-87bd-066bb541016d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:26.983075 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997eb9b6-40d5-4eee-87bd-066bb541016d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:26.990828 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 623/1264] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:21:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:26.996972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9238f32c-b3b2-4593-ae33-c75abbca157b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:27.062221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:27.095623 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9238f32c-b3b2-4593-ae33-c75abbca157b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:27.128298 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12bea6-1905-48e5-b586-b3b013f75e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:27.128909 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12bea6-1905-48e5-b586-b3b013f75e5b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:27.129185 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12bea6-1905-48e5-b586-b3b013f75e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:27.129452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12bea6-1905-48e5-b586-b3b013f75e5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:27.139575 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12bea6-1905-48e5-b586-b3b013f75e5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:27.141028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12bea6-1905-48e5-b586-b3b013f75e5b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:27.154940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-0c12bea6-1905-48e5-b586-b3b013f75e5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:27.154940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:27.154940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:27.154940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:27.154940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:27.154940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:27.154940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:27.154940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:27.154940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:27.154940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:27.154940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:27.154940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:27.154940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:27.154940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:27.154940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:27.154940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:27.156101 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:27.156101 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:27.156101 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:27.156101 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:27.156101 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:27.156101 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:27.156101 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:27.156101 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:27.156101 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:27.156101 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:27.156101 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:27.156101 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:27.156101 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:27.156101 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:27.156101 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:27.156101 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:27.156101 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:27.157238 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:27.157238 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:27.157238 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:21:27.157238 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 642/1265] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:27.162176 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997eb9b6-40d5-4eee-87bd-066bb541016d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:27.249878 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:27.269015 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997eb9b6-40d5-4eee-87bd-066bb541016d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:27.302296 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65445963-4231-42f7-b4af-beac6f6c49a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:27.302895 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65445963-4231-42f7-b4af-beac6f6c49a9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:27.303179 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65445963-4231-42f7-b4af-beac6f6c49a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:27.303456 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65445963-4231-42f7-b4af-beac6f6c49a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:27.304095 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65445963-4231-42f7-b4af-beac6f6c49a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:27.304402 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65445963-4231-42f7-b4af-beac6f6c49a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:27.305764 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65445963-4231-42f7-b4af-beac6f6c49a9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:27.316939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 624/1266] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:27.322659 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12bea6-1905-48e5-b586-b3b013f75e5b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:27.419211 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12bea6-1905-48e5-b586-b3b013f75e5b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:27.437618 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:27.451520 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c43a9-aa4a-4c9b-8201-016eb5f63d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:27.452136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c43a9-aa4a-4c9b-8201-016eb5f63d6c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:27.452413 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c43a9-aa4a-4c9b-8201-016eb5f63d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:27.452683 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c43a9-aa4a-4c9b-8201-016eb5f63d6c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:27.459955 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c43a9-aa4a-4c9b-8201-016eb5f63d6c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:27.461379 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c43a9-aa4a-4c9b-8201-016eb5f63d6c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:27.472751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-ce1c43a9-aa4a-4c9b-8201-016eb5f63d6c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:21:27.472751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:27.472751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:27.472751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:27.472751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:27.472751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:27.472751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:27.472751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:27.472751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:27.472751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:27.472751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:27.472751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:27.472751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:27.472751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:27.472751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:27.472751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:27.473916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:27.473916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:27.473916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:27.473916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:27.473916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:27.473916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:27.473916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:27.473916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:27.473916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:27.473916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:27.473916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:27.473916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:27.473916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:27.473916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:27.473916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:27.473916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:27.473916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:27.473916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:27.475197 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:27.475197 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:27.475197 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:27.475197 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:27.475197 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:27.475197 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:27.475197 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:27.475197 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:27.475197 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:21:27.475197 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:21:27.475197 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 643/1267] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:27.482582 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65445963-4231-42f7-b4af-beac6f6c49a9 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:27.592956 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65445963-4231-42f7-b4af-beac6f6c49a9 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:27.626170 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367a5fa-a6fb-4eec-8fcc-70405b7bacfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:27.626784 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367a5fa-a6fb-4eec-8fcc-70405b7bacfe None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:27.627060 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367a5fa-a6fb-4eec-8fcc-70405b7bacfe None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:27.627327 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367a5fa-a6fb-4eec-8fcc-70405b7bacfe None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:27.628043 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6367a5fa-a6fb-4eec-8fcc-70405b7bacfe None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:27.628341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6367a5fa-a6fb-4eec-8fcc-70405b7bacfe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:27.629750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6367a5fa-a6fb-4eec-8fcc-70405b7bacfe None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:27.638520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:27.647354 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 625/1268] 10.222.0.58 () {56 vars in 1143 bytes} [Thu Sep 1 16:21:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:27.653687 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c43a9-aa4a-4c9b-8201-016eb5f63d6c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:27.751550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c43a9-aa4a-4c9b-8201-016eb5f63d6c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:27.784467 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ac297-4c85-4053-9fd4-6d7e6cf34507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:27.785075 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ac297-4c85-4053-9fd4-6d7e6cf34507 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:27.785360 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ac297-4c85-4053-9fd4-6d7e6cf34507 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:27.785638 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ac297-4c85-4053-9fd4-6d7e6cf34507 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:27.796066 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ac297-4c85-4053-9fd4-6d7e6cf34507 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:27.797489 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ac297-4c85-4053-9fd4-6d7e6cf34507 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:27.811257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-3c0ac297-4c85-4053-9fd4-6d7e6cf34507 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:27.811257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:27.811257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:27.811257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:27.811257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:27.811257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:27.811257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:27.811257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:27.811257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:27.811257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:27.811257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:27.811257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:27.811257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:27.811257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:27.811257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:27.811257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:27.812424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:27.812424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:27.812424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:27.812424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:27.812424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:27.812424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:27.812424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:27.812424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:27.812424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:27.812424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:27.812424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:27.812424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:27.812424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:27.812424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:27.812424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:27.812424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:27.812424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:27.813394 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:27.813394 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:27.813394 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:21:27.813394 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 644/1269] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:27.818060 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367a5fa-a6fb-4eec-8fcc-70405b7bacfe None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:27.913291 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367a5fa-a6fb-4eec-8fcc-70405b7bacfe None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:27.945070 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d32aa-0452-4f9b-90ad-5342c99d2cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:27.945654 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d32aa-0452-4f9b-90ad-5342c99d2cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:27.945931 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d32aa-0452-4f9b-90ad-5342c99d2cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:27.946227 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d32aa-0452-4f9b-90ad-5342c99d2cb5 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:27.946864 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d32aa-0452-4f9b-90ad-5342c99d2cb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:27.947163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d32aa-0452-4f9b-90ad-5342c99d2cb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:27.948506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d32aa-0452-4f9b-90ad-5342c99d2cb5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:27.959616 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 626/1270] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:27.965708 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ac297-4c85-4053-9fd4-6d7e6cf34507 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:27.997733 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:28.064852 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ac297-4c85-4053-9fd4-6d7e6cf34507 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:28.097427 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed03e9b-fbf5-4c4a-8524-29d583b0ed92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:28.098043 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed03e9b-fbf5-4c4a-8524-29d583b0ed92 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:28.098359 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed03e9b-fbf5-4c4a-8524-29d583b0ed92 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:28.098636 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed03e9b-fbf5-4c4a-8524-29d583b0ed92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:28.105116 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed03e9b-fbf5-4c4a-8524-29d583b0ed92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:28.107213 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed03e9b-fbf5-4c4a-8524-29d583b0ed92 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:28.118865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-aed03e9b-fbf5-4c4a-8524-29d583b0ed92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:28.118865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:28.118865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:28.118865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:28.118865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:28.118865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:28.118865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:28.118865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:28.118865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:28.118865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:28.118865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:28.118865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:28.118865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:28.118865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:28.118865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:28.119940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:28.119940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:28.119940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:28.119940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:28.119940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:28.119940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:28.119940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:28.119940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:28.119940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:28.119940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:28.119940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:28.119940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:28.119940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:28.119940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:28.119940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:28.119940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:28.119940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:28.121065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:28.121065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:28.121065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:28.121065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:28.121065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:28.121065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:28.121065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:28.121065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:21:28.121065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 645/1271] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:28.127672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d32aa-0452-4f9b-90ad-5342c99d2cb5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:28.184942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:28.225202 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d32aa-0452-4f9b-90ad-5342c99d2cb5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:28.258667 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8dd0bf-4965-4cca-af2f-7f284cfa732d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:28.259265 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8dd0bf-4965-4cca-af2f-7f284cfa732d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:28.259551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8dd0bf-4965-4cca-af2f-7f284cfa732d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:28.259827 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8dd0bf-4965-4cca-af2f-7f284cfa732d None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:28.260519 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8dd0bf-4965-4cca-af2f-7f284cfa732d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:28.260828 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8dd0bf-4965-4cca-af2f-7f284cfa732d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:28.262897 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8dd0bf-4965-4cca-af2f-7f284cfa732d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:28.274552 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 627/1272] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:28.280742 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed03e9b-fbf5-4c4a-8524-29d583b0ed92 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:28.369975 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:28.379530 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed03e9b-fbf5-4c4a-8524-29d583b0ed92 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:28.412360 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4e998-72a2-4a39-b116-37ecba1b2955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:28.412995 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4e998-72a2-4a39-b116-37ecba1b2955 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:28.413270 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4e998-72a2-4a39-b116-37ecba1b2955 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:28.413540 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4e998-72a2-4a39-b116-37ecba1b2955 None admin] PATH_INFO: `/v3/projects/479530ced580426b85d1fc07af43417f/users/e12302cdb2854b0684179647a6dadf6e/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:28.415391 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de4e998-72a2-4a39-b116-37ecba1b2955 None admin] RBAC: Authorizing `identity:create_grant(project_id=479530ced580426b85d1fc07af43417f, user_id=e12302cdb2854b0684179647a6dadf6e, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:28.416796 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de4e998-72a2-4a39-b116-37ecba1b2955 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:28.429163 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 646/1273] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Sep 1 16:21:28 2022] PUT /identity/v3/projects/479530ced580426b85d1fc07af43417f/users/e12302cdb2854b0684179647a6dadf6e/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:21:30.327625 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833f20b-0e2e-4e66-a8dd-90116d64a399 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:30.328147 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833f20b-0e2e-4e66-a8dd-90116d64a399 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:30.328147 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833f20b-0e2e-4e66-a8dd-90116d64a399 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:30.329412 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 628/1274] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:30.335060 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd3a51-9469-4dc4-a34f-1acb69708c73 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:30.335507 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd3a51-9469-4dc4-a34f-1acb69708c73 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:30.335614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd3a51-9469-4dc4-a34f-1acb69708c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:30.353971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-0afd3a51-9469-4dc4-a34f-1acb69708c73 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:30.355776 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0afd3a51-9469-4dc4-a34f-1acb69708c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:30.419539 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 647/1275] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:21:30.704456 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3833f20b-0e2e-4e66-a8dd-90116d64a399 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:30.802829 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3833f20b-0e2e-4e66-a8dd-90116d64a399 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:30.819748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:30.836042 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9ed0f-e58d-4b9f-bce6-49ac1a3963f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:30.836655 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9ed0f-e58d-4b9f-bce6-49ac1a3963f2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:30.836936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9ed0f-e58d-4b9f-bce6-49ac1a3963f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:30.837206 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9ed0f-e58d-4b9f-bce6-49ac1a3963f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:30.846910 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9ed0f-e58d-4b9f-bce6-49ac1a3963f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:30.851647 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-4ad9ed0f-e58d-4b9f-bce6-49ac1a3963f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:21:30.851647 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:30.851647 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:30.851647 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:30.851647 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:30.851647 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:30.851647 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:30.851647 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:30.851647 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:30.851647 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:30.851647 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:30.851647 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:30.851647 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:30.851647 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:30.851647 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:30.851647 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:30.852793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:30.852793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:30.852793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:30.852793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:30.852793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:30.852793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:30.852793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:30.852793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:30.852793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:30.852793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:30.852793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:30.852793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:30.852793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:30.852793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:30.852793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:30.852793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:30.854031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:30.854031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:30.854031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:30.854031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:30.854031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:30.854031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:30.854031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:30.854031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:30.854031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:30.854031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:30.854031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:21:30.854031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:21:30.854031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 629/1276] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Sep 1 16:21:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:30.859077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd3a51-9469-4dc4-a34f-1acb69708c73 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:30.937661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:30.956034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd3a51-9469-4dc4-a34f-1acb69708c73 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:30.988141 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a7c01-d418-4e1a-990a-009db04b8711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:30.988727 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a7c01-d418-4e1a-990a-009db04b8711 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:30.989003 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a7c01-d418-4e1a-990a-009db04b8711 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:30.989268 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a7c01-d418-4e1a-990a-009db04b8711 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:30.989919 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a7c01-d418-4e1a-990a-009db04b8711 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:30.990238 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a7c01-d418-4e1a-990a-009db04b8711 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:30.991608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a7c01-d418-4e1a-990a-009db04b8711 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:30.999821 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 648/1277] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Sep 1 16:21:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:31.005611 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9ed0f-e58d-4b9f-bce6-49ac1a3963f2 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:31.104401 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9ed0f-e58d-4b9f-bce6-49ac1a3963f2 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:31.137026 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323decf6-5fde-42bc-8b12-21a92f9323a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:31.137644 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323decf6-5fde-42bc-8b12-21a92f9323a2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:31.137925 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323decf6-5fde-42bc-8b12-21a92f9323a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:31.138221 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323decf6-5fde-42bc-8b12-21a92f9323a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:31.147833 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323decf6-5fde-42bc-8b12-21a92f9323a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:31.149267 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323decf6-5fde-42bc-8b12-21a92f9323a2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:31.164823 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-323decf6-5fde-42bc-8b12-21a92f9323a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:31.164823 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:31.164823 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:31.164823 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:31.164823 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:31.164823 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:31.164823 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:31.164823 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:31.164823 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:31.164823 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:31.164823 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:31.164823 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:31.164823 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:31.164823 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:31.164823 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:31.164823 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:31.165972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:31.165972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:31.165972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:31.165972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:31.165972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:31.165972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:31.165972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:31.165972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:31.165972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:31.165972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:31.165972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:31.165972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:31.165972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:31.165972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:31.165972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:31.165972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:31.165972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:31.167832 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:31.167832 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:31.167832 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:21:31.167832 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 630/1278] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:31.172338 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a7c01-d418-4e1a-990a-009db04b8711 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:31.268873 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a7c01-d418-4e1a-990a-009db04b8711 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:31.301753 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4987ca44-8835-45bd-9fd1-2d13602aec1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:31.302373 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987ca44-8835-45bd-9fd1-2d13602aec1d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:31.302656 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987ca44-8835-45bd-9fd1-2d13602aec1d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:31.302929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987ca44-8835-45bd-9fd1-2d13602aec1d None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:31.303565 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4987ca44-8835-45bd-9fd1-2d13602aec1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:31.303867 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4987ca44-8835-45bd-9fd1-2d13602aec1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:31.305223 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4987ca44-8835-45bd-9fd1-2d13602aec1d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:31.316475 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 649/1279] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:31.322056 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323decf6-5fde-42bc-8b12-21a92f9323a2 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:31.352132 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:31.422861 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323decf6-5fde-42bc-8b12-21a92f9323a2 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:31.455290 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75dbe3c-dd0e-4ba2-8a79-5e6f36f213c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:31.455907 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75dbe3c-dd0e-4ba2-8a79-5e6f36f213c0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:31.456198 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75dbe3c-dd0e-4ba2-8a79-5e6f36f213c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:31.456476 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75dbe3c-dd0e-4ba2-8a79-5e6f36f213c0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:31.465966 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75dbe3c-dd0e-4ba2-8a79-5e6f36f213c0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:31.467423 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75dbe3c-dd0e-4ba2-8a79-5e6f36f213c0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:31.478236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-a75dbe3c-dd0e-4ba2-8a79-5e6f36f213c0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:21:31.478236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:31.478236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:31.478236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:31.478236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:31.478236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:31.478236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:31.478236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:31.478236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:31.478236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:31.478236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:31.478236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:31.478236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:31.478236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:31.478236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:31.478236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:31.479471 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:31.479471 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:31.479471 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:31.479471 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:31.479471 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:31.479471 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:31.479471 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:31.479471 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:31.479471 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:31.479471 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:31.479471 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:31.479471 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:31.479471 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:31.479471 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:31.479471 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:31.479471 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:31.479471 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:31.479471 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:31.481332 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:31.481332 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:31.481332 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:31.481332 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:31.481332 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:31.481332 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:31.481332 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:31.481332 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:31.481332 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:21:31.481332 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:21:31.481332 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 631/1280] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:31.485617 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4987ca44-8835-45bd-9fd1-2d13602aec1d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:31.489343 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:31.595355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4987ca44-8835-45bd-9fd1-2d13602aec1d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:31.627920 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9cdd2e-9b9f-4281-b36a-911dde181b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:31.628509 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cdd2e-9b9f-4281-b36a-911dde181b0b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:31.628783 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cdd2e-9b9f-4281-b36a-911dde181b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:31.629053 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cdd2e-9b9f-4281-b36a-911dde181b0b None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:31.629771 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9cdd2e-9b9f-4281-b36a-911dde181b0b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:31.630071 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9cdd2e-9b9f-4281-b36a-911dde181b0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:31.631507 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9cdd2e-9b9f-4281-b36a-911dde181b0b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:31.648404 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 650/1281] 10.222.0.58 () {56 vars in 1143 bytes} [Thu Sep 1 16:21:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:31.654232 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75dbe3c-dd0e-4ba2-8a79-5e6f36f213c0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:31.741752 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:31.751542 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75dbe3c-dd0e-4ba2-8a79-5e6f36f213c0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:31.784411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639468b-786b-4a04-a629-e8a10b3f3618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:31.784996 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639468b-786b-4a04-a629-e8a10b3f3618 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:31.785272 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639468b-786b-4a04-a629-e8a10b3f3618 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:31.785537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639468b-786b-4a04-a629-e8a10b3f3618 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:31.857544 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6639468b-786b-4a04-a629-e8a10b3f3618 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:31.859061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6639468b-786b-4a04-a629-e8a10b3f3618 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:31.957383 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:31.968415 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 632/1282] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Sep 1 16:21:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:21:31.974287 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9cdd2e-9b9f-4281-b36a-911dde181b0b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:32.036712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:32.077187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9cdd2e-9b9f-4281-b36a-911dde181b0b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:32.111387 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f0c04-ee7d-4f7c-9a00-73a685955be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:32.111946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f0c04-ee7d-4f7c-9a00-73a685955be1 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:32.112229 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f0c04-ee7d-4f7c-9a00-73a685955be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:32.112504 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f0c04-ee7d-4f7c-9a00-73a685955be1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:32.122313 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f0c04-ee7d-4f7c-9a00-73a685955be1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:32.123750 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f0c04-ee7d-4f7c-9a00-73a685955be1 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:32.138319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-8a0f0c04-ee7d-4f7c-9a00-73a685955be1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:32.138319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:32.138319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:32.138319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:32.138319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:32.138319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:32.138319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:32.138319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:32.138319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:32.138319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:32.138319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:32.138319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:32.138319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:32.138319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:32.138319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:32.138319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:32.139451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:32.139451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:32.139451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:32.139451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:32.139451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:32.139451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:32.139451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:32.139451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:32.139451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:32.139451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:32.139451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:32.139451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:32.139451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:32.139451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:32.139451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:32.139451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:32.139451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:32.140740 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:32.140740 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:32.140740 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:21:32.140740 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 651/1283] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:32.145205 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639468b-786b-4a04-a629-e8a10b3f3618 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:32.263570 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639468b-786b-4a04-a629-e8a10b3f3618 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:32.297495 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269871a-f1bb-4068-b4c4-87016a4b7920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:32.298083 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269871a-f1bb-4068-b4c4-87016a4b7920 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:32.298390 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269871a-f1bb-4068-b4c4-87016a4b7920 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:32.298660 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269871a-f1bb-4068-b4c4-87016a4b7920 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:32.299292 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269871a-f1bb-4068-b4c4-87016a4b7920 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:32.299593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269871a-f1bb-4068-b4c4-87016a4b7920 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:32.300953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269871a-f1bb-4068-b4c4-87016a4b7920 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:32.311661 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 633/1284] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:32.317352 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f0c04-ee7d-4f7c-9a00-73a685955be1 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:32.414431 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f0c04-ee7d-4f7c-9a00-73a685955be1 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:32.427826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:32.447455 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f4e8d-7aa1-41af-bc98-cf81483cf172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:32.448065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f4e8d-7aa1-41af-bc98-cf81483cf172 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:32.448831 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f4e8d-7aa1-41af-bc98-cf81483cf172 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:32.449099 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f4e8d-7aa1-41af-bc98-cf81483cf172 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:32.455492 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f4e8d-7aa1-41af-bc98-cf81483cf172 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:32.456911 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f4e8d-7aa1-41af-bc98-cf81483cf172 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:32.468341 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-830f4e8d-7aa1-41af-bc98-cf81483cf172 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:32.468341 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:32.468341 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:32.468341 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:32.468341 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:32.468341 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:32.468341 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:32.468341 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:32.468341 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:32.468341 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:32.468341 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:32.468341 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:32.468341 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:32.468341 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:32.468341 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:32.469396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:32.469396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:32.469396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:32.469396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:32.469396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:32.469396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:32.469396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:32.469396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:32.469396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:32.469396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:32.469396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:32.469396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:32.469396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:32.469396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:32.469396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:32.469396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:32.469396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:32.470570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:32.470570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:32.470570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:32.470570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:32.470570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:32.470570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:32.470570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:32.470570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Sep 01 16:21:32.470570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 652/1285] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:32.475668 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269871a-f1bb-4068-b4c4-87016a4b7920 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:32.528784 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:32.587800 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269871a-f1bb-4068-b4c4-87016a4b7920 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:32.625448 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f932532-9ba9-456d-a2d1-041aeffd8022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:32.626044 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f932532-9ba9-456d-a2d1-041aeffd8022 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:32.626359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f932532-9ba9-456d-a2d1-041aeffd8022 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:32.626640 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f932532-9ba9-456d-a2d1-041aeffd8022 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:32.627334 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f932532-9ba9-456d-a2d1-041aeffd8022 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:32.627642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f932532-9ba9-456d-a2d1-041aeffd8022 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:32.629065 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f932532-9ba9-456d-a2d1-041aeffd8022 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:32.641177 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 634/1286] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:32.647314 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f4e8d-7aa1-41af-bc98-cf81483cf172 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:32.743901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f4e8d-7aa1-41af-bc98-cf81483cf172 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:32.776886 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64961204-5d17-45f5-8eb8-8ceaa54e011a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:32.777470 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64961204-5d17-45f5-8eb8-8ceaa54e011a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:32.777747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64961204-5d17-45f5-8eb8-8ceaa54e011a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:32.778019 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64961204-5d17-45f5-8eb8-8ceaa54e011a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:32.778772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64961204-5d17-45f5-8eb8-8ceaa54e011a None admin] RBAC: Adding query filter params (role.id=ff293acb8551463aafe6970230042b89, scope.project.id=479530ced580426b85d1fc07af43417f, user.id=e12302cdb2854b0684179647a6dadf6e) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:32.779080 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64961204-5d17-45f5-8eb8-8ceaa54e011a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:32.780509 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64961204-5d17-45f5-8eb8-8ceaa54e011a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:32.785983 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 653/1287] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Sep 1 16:21:32 2022] GET /identity/v3/role_assignments?user.id=e12302cdb2854b0684179647a6dadf6e&scope.project.id=479530ced580426b85d1fc07af43417f&role.id=ff293acb8551463aafe6970230042b89 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:34.700472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbc123-cd33-41a3-8ad6-bdd054ee8422 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:34.701000 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbc123-cd33-41a3-8ad6-bdd054ee8422 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:34.701000 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbc123-cd33-41a3-8ad6-bdd054ee8422 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:34.702369 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 635/1288] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:34.707455 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476513e1-3678-4ec4-86d8-c3ccdd815aa3 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:34.707899 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476513e1-3678-4ec4-86d8-c3ccdd815aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:34.707995 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476513e1-3678-4ec4-86d8-c3ccdd815aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:34.726391 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-476513e1-3678-4ec4-86d8-c3ccdd815aa3 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:34.727559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-476513e1-3678-4ec4-86d8-c3ccdd815aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:34.756578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:34.792923 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 654/1289] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:21:35.056723 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bbc123-cd33-41a3-8ad6-bdd054ee8422 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:35.159041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bbc123-cd33-41a3-8ad6-bdd054ee8422 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:35.169411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:35.192031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc69661-8bd3-4557-996e-c5b2705e4b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:35.192646 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc69661-8bd3-4557-996e-c5b2705e4b79 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:35.192925 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc69661-8bd3-4557-996e-c5b2705e4b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:35.193197 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc69661-8bd3-4557-996e-c5b2705e4b79 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:35.193777 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc69661-8bd3-4557-996e-c5b2705e4b79 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:35.195190 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc69661-8bd3-4557-996e-c5b2705e4b79 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:35.204193 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-afc69661-8bd3-4557-996e-c5b2705e4b79 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:21:35.204193 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:35.204193 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:21:35.204193 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:21:35.204193 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:21:35.204193 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:21:35.204193 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:21:35.204193 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:21:35.204193 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:21:35.204193 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:21:35.204193 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:21:35.204193 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:21:35.204193 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:35.204193 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:35.204193 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:35.204193 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:35.204193 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:35.205557 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:35.205557 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:35.205557 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:35.205557 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:35.205557 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:35.205557 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:35.205557 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:21:35.205557 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:21:35.205557 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:21:35.205557 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:21:35.205557 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:35.205557 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:35.205557 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:35.205557 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:35.205557 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:35.205557 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:35.206671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:35.206671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:35.206671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:35.206671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:35.206671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:35.206671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:35.206671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:35.206671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:35.206671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:35.206671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:35.206671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:21:35.206671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:21:35.206671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:21:35.206671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:21:35.206671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 636/1290] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Sep 1 16:21:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:35.211214 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476513e1-3678-4ec4-86d8-c3ccdd815aa3 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:35.306725 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476513e1-3678-4ec4-86d8-c3ccdd815aa3 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:35.339893 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226b17a-743d-4438-a157-803f625010ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:35.340483 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226b17a-743d-4438-a157-803f625010ad None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:35.340761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226b17a-743d-4438-a157-803f625010ad None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:35.341032 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226b17a-743d-4438-a157-803f625010ad None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:35.341655 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f226b17a-743d-4438-a157-803f625010ad None admin] RBAC: Adding query filter params (name=placement) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:35.341953 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f226b17a-743d-4438-a157-803f625010ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:35.343339 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f226b17a-743d-4438-a157-803f625010ad None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:35.347932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 655/1291] 10.222.0.58 () {56 vars in 1116 bytes} [Thu Sep 1 16:21:35 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:35.353607 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc69661-8bd3-4557-996e-c5b2705e4b79 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:35.450515 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc69661-8bd3-4557-996e-c5b2705e4b79 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:35.482876 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c01dc1-3c96-4943-9fd8-85ef5747f3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:35.483468 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c01dc1-3c96-4943-9fd8-85ef5747f3b7 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:35.483752 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c01dc1-3c96-4943-9fd8-85ef5747f3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:35.484029 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c01dc1-3c96-4943-9fd8-85ef5747f3b7 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:35.484656 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c01dc1-3c96-4943-9fd8-85ef5747f3b7 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:35.484961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c01dc1-3c96-4943-9fd8-85ef5747f3b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:35.486348 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c01dc1-3c96-4943-9fd8-85ef5747f3b7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:35.490621 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 637/1292] 10.222.0.58 () {56 vars in 1116 bytes} [Thu Sep 1 16:21:35 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:37.349626 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb05387-e020-400d-9a85-6f9abc10ee2c None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:37.350139 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb05387-e020-400d-9a85-6f9abc10ee2c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:37.350139 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb05387-e020-400d-9a85-6f9abc10ee2c None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:37.351432 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 656/1293] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:37.357315 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d139f6-c0d0-4963-bfe5-f981fd873026 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:37.357704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d139f6-c0d0-4963-bfe5-f981fd873026 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:37.357859 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d139f6-c0d0-4963-bfe5-f981fd873026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:37.375506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-13d139f6-c0d0-4963-bfe5-f981fd873026 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:37.376674 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-13d139f6-c0d0-4963-bfe5-f981fd873026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:37.405685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:37.441390 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 638/1294] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:21:37.719164 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb05387-e020-400d-9a85-6f9abc10ee2c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:37.749892 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:37.818913 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb05387-e020-400d-9a85-6f9abc10ee2c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:37.852046 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef6a57-60ed-47ea-8f64-39176abcf4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:37.852633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef6a57-60ed-47ea-8f64-39176abcf4cb None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:37.852909 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef6a57-60ed-47ea-8f64-39176abcf4cb None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:37.853177 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef6a57-60ed-47ea-8f64-39176abcf4cb None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:37.853828 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef6a57-60ed-47ea-8f64-39176abcf4cb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:37.855227 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef6a57-60ed-47ea-8f64-39176abcf4cb None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:37.868914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 657/1295] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Sep 1 16:21:37 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:21:39.791175 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411db3a-fc78-4fb8-97c3-c4096bba1100 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:39.792222 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411db3a-fc78-4fb8-97c3-c4096bba1100 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:39.792222 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411db3a-fc78-4fb8-97c3-c4096bba1100 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:39.793566 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 639/1296] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:39.798727 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf7062-0cca-4604-a8e0-5c9f0c2586ba None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:39.799123 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf7062-0cca-4604-a8e0-5c9f0c2586ba None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:39.799281 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf7062-0cca-4604-a8e0-5c9f0c2586ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:39.817775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-07bf7062-0cca-4604-a8e0-5c9f0c2586ba None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:39.819097 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-07bf7062-0cca-4604-a8e0-5c9f0c2586ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:39.889940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 658/1297] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:39 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:21:40.162178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411db3a-fc78-4fb8-97c3-c4096bba1100 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:40.259025 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411db3a-fc78-4fb8-97c3-c4096bba1100 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:40.292325 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0f640-8c47-4adc-99ee-07d57856a577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:40.292931 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0f640-8c47-4adc-99ee-07d57856a577 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:40.293208 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0f640-8c47-4adc-99ee-07d57856a577 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:40.293498 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0f640-8c47-4adc-99ee-07d57856a577 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:40.294076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0f640-8c47-4adc-99ee-07d57856a577 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:40.295453 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0f640-8c47-4adc-99ee-07d57856a577 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:40.303917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-5ef0f640-8c47-4adc-99ee-07d57856a577 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:21:40.303917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:40.303917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:21:40.303917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:21:40.303917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:21:40.303917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:21:40.303917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:21:40.303917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:21:40.303917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:21:40.303917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:21:40.303917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:21:40.303917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:21:40.303917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:40.303917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:40.303917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:40.303917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:40.303917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:40.305138 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:40.305138 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:40.305138 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:40.305138 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:40.305138 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:40.305138 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:40.305138 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:21:40.305138 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:21:40.305138 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:21:40.305138 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:21:40.305138 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:40.305138 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:40.305138 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:40.305138 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:40.305138 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:40.305138 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:40.306308 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:40.306308 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:40.306308 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:40.306308 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:40.306308 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:40.306308 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:40.306308 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:40.306308 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:40.306308 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:40.306308 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:40.306308 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:21:40.306308 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:21:40.306308 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:21:40.306308 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:21:40.306308 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 640/1298] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Sep 1 16:21:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:40.310938 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf7062-0cca-4604-a8e0-5c9f0c2586ba None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:40.328739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:40.407581 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf7062-0cca-4604-a8e0-5c9f0c2586ba None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:40.448962 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4fedf6-6537-42aa-b75a-0ea39f9ac8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:40.449560 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4fedf6-6537-42aa-b75a-0ea39f9ac8f1 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:40.449840 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4fedf6-6537-42aa-b75a-0ea39f9ac8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:40.450114 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4fedf6-6537-42aa-b75a-0ea39f9ac8f1 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:40.450771 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4fedf6-6537-42aa-b75a-0ea39f9ac8f1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:40.451072 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4fedf6-6537-42aa-b75a-0ea39f9ac8f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:40.452429 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4fedf6-6537-42aa-b75a-0ea39f9ac8f1 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:40.457356 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 659/1299] 10.222.0.58 () {56 vars in 1116 bytes} [Thu Sep 1 16:21:40 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:40.464048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0f640-8c47-4adc-99ee-07d57856a577 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:40.494043 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:40.576644 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0f640-8c47-4adc-99ee-07d57856a577 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:40.609627 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579188ca-a07d-4ec1-8e48-924d20d90782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:40.610249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579188ca-a07d-4ec1-8e48-924d20d90782 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:40.610536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579188ca-a07d-4ec1-8e48-924d20d90782 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:40.610813 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579188ca-a07d-4ec1-8e48-924d20d90782 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:40.611513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579188ca-a07d-4ec1-8e48-924d20d90782 None admin] RBAC: Adding query filter params (interface=public, service_id=4885aa48a1004f4cb6e85b62e10b675f, region_id=RegionOne) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:40.611820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579188ca-a07d-4ec1-8e48-924d20d90782 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:40.613183 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579188ca-a07d-4ec1-8e48-924d20d90782 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:40.618050 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 641/1300] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Sep 1 16:21:40 2022] GET /identity/v3/endpoints?service_id=4885aa48a1004f4cb6e85b62e10b675f&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:40.623801 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4fedf6-6537-42aa-b75a-0ea39f9ac8f1 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:40.701834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:40.719811 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4fedf6-6537-42aa-b75a-0ea39f9ac8f1 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:40.752518 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69728ff3-7810-45e8-8d00-6ccd6a75d187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:40.753105 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69728ff3-7810-45e8-8d00-6ccd6a75d187 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:40.753380 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69728ff3-7810-45e8-8d00-6ccd6a75d187 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:40.753646 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69728ff3-7810-45e8-8d00-6ccd6a75d187 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:40.754281 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69728ff3-7810-45e8-8d00-6ccd6a75d187 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:40.755614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69728ff3-7810-45e8-8d00-6ccd6a75d187 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:40.762419 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 660/1301] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Sep 1 16:21:40 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:21:42.680596 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab4050-4bb5-4cf7-bd36-5d33b13ea0aa None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:42.681161 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab4050-4bb5-4cf7-bd36-5d33b13ea0aa None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:42.681161 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab4050-4bb5-4cf7-bd36-5d33b13ea0aa None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:42.682521 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 642/1302] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:42.687510 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9defe0c-7a7c-4737-92ff-61503a4da92a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:42.687917 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9defe0c-7a7c-4737-92ff-61503a4da92a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:42.688080 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9defe0c-7a7c-4737-92ff-61503a4da92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:42.706157 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-c9defe0c-7a7c-4737-92ff-61503a4da92a None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:42.707359 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c9defe0c-7a7c-4737-92ff-61503a4da92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:42.771204 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 661/1303] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:42 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:21:43.045184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab4050-4bb5-4cf7-bd36-5d33b13ea0aa None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:43.144157 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:43.152591 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab4050-4bb5-4cf7-bd36-5d33b13ea0aa None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:43.187542 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca17c8c-8b67-45e3-8e05-83c8ccb2c59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:43.188165 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca17c8c-8b67-45e3-8e05-83c8ccb2c59b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:43.188447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca17c8c-8b67-45e3-8e05-83c8ccb2c59b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:43.188721 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca17c8c-8b67-45e3-8e05-83c8ccb2c59b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:43.189308 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca17c8c-8b67-45e3-8e05-83c8ccb2c59b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:43.190717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca17c8c-8b67-45e3-8e05-83c8ccb2c59b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:43.199333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-6ca17c8c-8b67-45e3-8e05-83c8ccb2c59b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:21:43.199333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:43.199333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:21:43.199333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:21:43.199333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:21:43.199333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:21:43.199333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:21:43.199333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:21:43.199333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:21:43.199333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:21:43.199333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:21:43.199333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:21:43.199333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:43.199333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:43.199333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:43.199333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:43.199333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:43.200511 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:43.200511 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:43.200511 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:43.200511 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:43.200511 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:43.200511 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:43.200511 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:21:43.200511 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:21:43.200511 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:21:43.200511 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:21:43.200511 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:43.200511 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:43.200511 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:43.200511 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:43.200511 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:43.200511 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:43.201386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:43.201386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:43.201386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:43.201386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:43.201386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:43.201386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:43.201386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:43.201386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:43.201386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:43.201386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:43.201386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:21:43.201386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:21:43.201386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:21:43.201386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Sep 01 16:21:43.201386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 643/1304] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Sep 1 16:21:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:43.206622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9defe0c-7a7c-4737-92ff-61503a4da92a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:43.280049 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:43.302844 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9defe0c-7a7c-4737-92ff-61503a4da92a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:43.335206 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e4e0b0-d061-4672-8d90-e7fd7cca8bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:43.335797 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4e0b0-d061-4672-8d90-e7fd7cca8bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:43.336075 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4e0b0-d061-4672-8d90-e7fd7cca8bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:43.336344 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4e0b0-d061-4672-8d90-e7fd7cca8bc4 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:43.336968 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e4e0b0-d061-4672-8d90-e7fd7cca8bc4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:43.337269 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e4e0b0-d061-4672-8d90-e7fd7cca8bc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:43.339159 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e4e0b0-d061-4672-8d90-e7fd7cca8bc4 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:43.344042 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 662/1305] 10.222.0.58 () {56 vars in 1116 bytes} [Thu Sep 1 16:21:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:43.349822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca17c8c-8b67-45e3-8e05-83c8ccb2c59b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:43.449761 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca17c8c-8b67-45e3-8e05-83c8ccb2c59b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:43.482929 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97071876-7f21-4be0-aa9a-b2e11dc112fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:43.483521 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97071876-7f21-4be0-aa9a-b2e11dc112fd None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:43.483806 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97071876-7f21-4be0-aa9a-b2e11dc112fd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:43.484081 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97071876-7f21-4be0-aa9a-b2e11dc112fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:43.484741 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97071876-7f21-4be0-aa9a-b2e11dc112fd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:43.486110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97071876-7f21-4be0-aa9a-b2e11dc112fd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:43.505096 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 644/1306] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:21:43 2022] POST /identity/v3/endpoints => generated 331 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:21:53.341342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=30 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:53.502415 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:47.361504 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928942e-f1bf-4fd0-bd0d-32dd1baf80e7 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:47.362434 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928942e-f1bf-4fd0-bd0d-32dd1baf80e7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:47.362861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928942e-f1bf-4fd0-bd0d-32dd1baf80e7 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:47.368786 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 663/1307] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:22:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:47.370227 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd18e9-9983-426c-a7a1-76f99e1bbe4c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:47.370643 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd18e9-9983-426c-a7a1-76f99e1bbe4c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:47.371022 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd18e9-9983-426c-a7a1-76f99e1bbe4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:47.523777 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-52fd18e9-9983-426c-a7a1-76f99e1bbe4c None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:47.525015 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-52fd18e9-9983-426c-a7a1-76f99e1bbe4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:47.770228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 645/1308] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:22:48.067340 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0835732-c22e-4495-a626-9c24268d3f0d None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:48.068027 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0835732-c22e-4495-a626-9c24268d3f0d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:48.068553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0835732-c22e-4495-a626-9c24268d3f0d None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:48.070689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 664/1309] 10.222.0.58 () {54 vars in 965 bytes} [Thu Sep 1 16:22:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:48.077024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2817e32-e7e7-45ad-8d32-45faadb705ce None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:48.077346 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2817e32-e7e7-45ad-8d32-45faadb705ce None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:48.077593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2817e32-e7e7-45ad-8d32-45faadb705ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:48.171111 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e2817e32-e7e7-45ad-8d32-45faadb705ce None None] MFA Rules not processed for user `943fae29c5dc41f2911bbe051c74a361`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:48.172493 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e2817e32-e7e7-45ad-8d32-45faadb705ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:48.195880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:48.290415 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 646/1310] 10.222.0.58 () {58 vars in 1062 bytes} [Thu Sep 1 16:22:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:22:48.291698 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0835732-c22e-4495-a626-9c24268d3f0d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:48.293635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f0835732-c22e-4495-a626-9c24268d3f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:48.435751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0835732-c22e-4495-a626-9c24268d3f0d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:48.518818 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafc5b80-d78e-4810-80eb-cd134e5ce950 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:48.519432 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc5b80-d78e-4810-80eb-cd134e5ce950 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:48.519716 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc5b80-d78e-4810-80eb-cd134e5ce950 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:48.520600 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc5b80-d78e-4810-80eb-cd134e5ce950 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:48.522536 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-dafc5b80-d78e-4810-80eb-cd134e5ce950 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:48.580829 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:48.650812 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafc5b80-d78e-4810-80eb-cd134e5ce950 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:48.652484 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafc5b80-d78e-4810-80eb-cd134e5ce950 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:48.677268 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 665/1311] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:22:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:22:50.901730 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9dd53c-53de-4634-97ed-69f059ac09ad None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:50.902344 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9dd53c-53de-4634-97ed-69f059ac09ad None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:50.902344 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9dd53c-53de-4634-97ed-69f059ac09ad None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:50.903666 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 647/1312] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:50.909493 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ea44f-597d-41f2-94a4-277f5d4e5e8b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:50.910013 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ea44f-597d-41f2-94a4-277f5d4e5e8b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:50.910118 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ea44f-597d-41f2-94a4-277f5d4e5e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:51.000452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-670ea44f-597d-41f2-94a4-277f5d4e5e8b None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:51.002034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-670ea44f-597d-41f2-94a4-277f5d4e5e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:51.104513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 666/1313] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:22:51.395609 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9dd53c-53de-4634-97ed-69f059ac09ad None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:51.397121 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ef9dd53c-53de-4634-97ed-69f059ac09ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:51.495699 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9dd53c-53de-4634-97ed-69f059ac09ad None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:51.564476 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:51.567662 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfbff6d-1a48-4df1-ba47-03f457e6dd77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:51.568474 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbff6d-1a48-4df1-ba47-03f457e6dd77 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:51.568924 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbff6d-1a48-4df1-ba47-03f457e6dd77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:51.569374 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbff6d-1a48-4df1-ba47-03f457e6dd77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:51.570988 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0bfbff6d-1a48-4df1-ba47-03f457e6dd77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:51.678036 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfbff6d-1a48-4df1-ba47-03f457e6dd77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:51.679564 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfbff6d-1a48-4df1-ba47-03f457e6dd77 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:51.697605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 648/1314] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:22:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:22:53.905847 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c30648-2489-4516-9767-156b25d7b73e None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:53.906398 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c30648-2489-4516-9767-156b25d7b73e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:53.906398 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c30648-2489-4516-9767-156b25d7b73e None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:53.908015 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 667/1315] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:22:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:53.913538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6852d-c051-423d-81d5-3464e69818f1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:53.913918 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6852d-c051-423d-81d5-3464e69818f1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:53.914091 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6852d-c051-423d-81d5-3464e69818f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:53.979682 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-53d6852d-c051-423d-81d5-3464e69818f1 None None] MFA Rules not processed for user `3c9bc22bd8da47e98a53e9c6e78236ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:53.980804 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-53d6852d-c051-423d-81d5-3464e69818f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:54.030953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:54.077829 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 649/1316] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:53 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:22:54.381428 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee2629-6cc1-4564-b222-6cb41cbeb926 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:54.381713 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee2629-6cc1-4564-b222-6cb41cbeb926 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:54.381995 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee2629-6cc1-4564-b222-6cb41cbeb926 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:54.383362 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 668/1317] 10.222.0.58 () {54 vars in 965 bytes} [Thu Sep 1 16:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:54.389029 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b80e1-4a62-43de-b3d1-c1ba6eeaf9ec None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:54.389314 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b80e1-4a62-43de-b3d1-c1ba6eeaf9ec None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:54.389592 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b80e1-4a62-43de-b3d1-c1ba6eeaf9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:54.465144 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3f2b80e1-4a62-43de-b3d1-c1ba6eeaf9ec None None] MFA Rules not processed for user `943fae29c5dc41f2911bbe051c74a361`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:54.466394 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3f2b80e1-4a62-43de-b3d1-c1ba6eeaf9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:54.543964 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 650/1318] 10.222.0.58 () {58 vars in 1062 bytes} [Thu Sep 1 16:22:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:22:54.553049 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee2629-6cc1-4564-b222-6cb41cbeb926 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:54.554901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c7ee2629-6cc1-4564-b222-6cb41cbeb926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:54.564471 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:54.657577 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee2629-6cc1-4564-b222-6cb41cbeb926 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:54.728401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce94e46-13dd-4502-a6e8-92367eb9e2b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:54.728994 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce94e46-13dd-4502-a6e8-92367eb9e2b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:54.729272 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce94e46-13dd-4502-a6e8-92367eb9e2b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:54.729540 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce94e46-13dd-4502-a6e8-92367eb9e2b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:54.731169 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6ce94e46-13dd-4502-a6e8-92367eb9e2b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:54.791910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:54.837842 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce94e46-13dd-4502-a6e8-92367eb9e2b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:54.839288 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce94e46-13dd-4502-a6e8-92367eb9e2b8 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:54.856937 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 669/1319] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:22:54 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:22:57.701134 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3ad1a-304e-4008-aeb4-7ea7d7428601 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:57.701702 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3ad1a-304e-4008-aeb4-7ea7d7428601 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:57.701702 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3ad1a-304e-4008-aeb4-7ea7d7428601 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:57.703112 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 651/1320] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:57.708850 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1befb3c-8523-46fe-af70-7ace1cded324 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:57.709282 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1befb3c-8523-46fe-af70-7ace1cded324 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:57.709399 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1befb3c-8523-46fe-af70-7ace1cded324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:57.774042 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f1befb3c-8523-46fe-af70-7ace1cded324 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:57.775282 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f1befb3c-8523-46fe-af70-7ace1cded324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:57.874741 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 670/1321] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:22:58.189567 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e3ad1a-304e-4008-aeb4-7ea7d7428601 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:58.191190 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-71e3ad1a-304e-4008-aeb4-7ea7d7428601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:58.247433 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:58.305968 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e3ad1a-304e-4008-aeb4-7ea7d7428601 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:58.384836 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d59c92-ca86-4468-82ed-8f76d4da0d30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:58.385436 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d59c92-ca86-4468-82ed-8f76d4da0d30 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:58.385720 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d59c92-ca86-4468-82ed-8f76d4da0d30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:58.385995 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d59c92-ca86-4468-82ed-8f76d4da0d30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:58.387652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-21d59c92-ca86-4468-82ed-8f76d4da0d30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:58.493965 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:58.508364 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d59c92-ca86-4468-82ed-8f76d4da0d30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:58.510762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d59c92-ca86-4468-82ed-8f76d4da0d30 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:58.536860 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 652/1322] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:22:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:23:01.191110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb47d6-b882-46ac-8feb-c7c2a79b0af0 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:01.191666 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb47d6-b882-46ac-8feb-c7c2a79b0af0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:01.191666 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb47d6-b882-46ac-8feb-c7c2a79b0af0 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:01.193026 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 671/1323] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:01.198277 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ca4c6-5af0-4177-bf53-8cd9e691cd6c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:01.198663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ca4c6-5af0-4177-bf53-8cd9e691cd6c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:01.198838 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ca4c6-5af0-4177-bf53-8cd9e691cd6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:01.276354 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7c2ca4c6-5af0-4177-bf53-8cd9e691cd6c None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:01.277567 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7c2ca4c6-5af0-4177-bf53-8cd9e691cd6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:01.379050 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 653/1324] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:23:01.662988 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb47d6-b882-46ac-8feb-c7c2a79b0af0 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:01.664599 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0fcb47d6-b882-46ac-8feb-c7c2a79b0af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:01.683381 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:01.767703 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb47d6-b882-46ac-8feb-c7c2a79b0af0 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:01.848053 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c62e80-3846-4834-9a8d-968a60208bc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:01.848650 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c62e80-3846-4834-9a8d-968a60208bc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:01.848930 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c62e80-3846-4834-9a8d-968a60208bc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:01.849203 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c62e80-3846-4834-9a8d-968a60208bc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:01.850860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-23c62e80-3846-4834-9a8d-968a60208bc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:01.921530 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:01.961541 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c62e80-3846-4834-9a8d-968a60208bc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:01.963685 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c62e80-3846-4834-9a8d-968a60208bc2 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:01.982545 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 672/1325] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:23:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:23:03.983309 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0f8a4-a38b-4940-92ed-0238db739c36 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:03.983870 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0f8a4-a38b-4940-92ed-0238db739c36 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:03.983870 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0f8a4-a38b-4940-92ed-0238db739c36 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:03.985218 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 654/1326] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:03.995141 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438212d3-60c9-43bb-ac1c-4ad94464cd0a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:03.995877 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438212d3-60c9-43bb-ac1c-4ad94464cd0a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:03.996146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438212d3-60c9-43bb-ac1c-4ad94464cd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:04.065644 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-438212d3-60c9-43bb-ac1c-4ad94464cd0a None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:04.067009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-438212d3-60c9-43bb-ac1c-4ad94464cd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:04.162950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 673/1327] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:23:04.463135 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0f8a4-a38b-4940-92ed-0238db739c36 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:04.464653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c8e0f8a4-a38b-4940-92ed-0238db739c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:04.528537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:04.571608 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0f8a4-a38b-4940-92ed-0238db739c36 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:04.655653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d3eea-1531-4e03-b23d-406386ae1024 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:04.656266 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d3eea-1531-4e03-b23d-406386ae1024 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:04.656548 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d3eea-1531-4e03-b23d-406386ae1024 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:04.657076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d3eea-1531-4e03-b23d-406386ae1024 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:04.658721 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fd5d3eea-1531-4e03-b23d-406386ae1024 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:04.767081 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:04.769435 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d3eea-1531-4e03-b23d-406386ae1024 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:04.771109 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d3eea-1531-4e03-b23d-406386ae1024 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:04.790134 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 655/1328] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:23:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:23:06.760855 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc6896-4cdf-4f53-93e5-f4737954bf36 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:06.761406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc6896-4cdf-4f53-93e5-f4737954bf36 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:06.761406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc6896-4cdf-4f53-93e5-f4737954bf36 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:06.762952 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 674/1329] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:06.768209 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef647f9-bb38-4022-88a0-a12c9cc916d1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:06.769163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef647f9-bb38-4022-88a0-a12c9cc916d1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:06.769368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef647f9-bb38-4022-88a0-a12c9cc916d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:06.833972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-aef647f9-bb38-4022-88a0-a12c9cc916d1 None None] MFA Rules not processed for user `3c9bc22bd8da47e98a53e9c6e78236ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:06.835185 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-aef647f9-bb38-4022-88a0-a12c9cc916d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:06.933752 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 656/1330] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:06 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:23:07.214874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc6896-4cdf-4f53-93e5-f4737954bf36 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:07.216384 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-73bc6896-4cdf-4f53-93e5-f4737954bf36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:07.246676 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:07.333211 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc6896-4cdf-4f53-93e5-f4737954bf36 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:07.407056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c834a-82e2-4a4b-9835-47d6cd69adb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:07.407575 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c834a-82e2-4a4b-9835-47d6cd69adb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:07.407850 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c834a-82e2-4a4b-9835-47d6cd69adb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:07.408125 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c834a-82e2-4a4b-9835-47d6cd69adb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:07.410456 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-fa0c834a-82e2-4a4b-9835-47d6cd69adb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:07.492691 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:07.524993 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c834a-82e2-4a4b-9835-47d6cd69adb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:07.526446 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c834a-82e2-4a4b-9835-47d6cd69adb4 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:07.546402 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 675/1331] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:23:07 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:23:09.687202 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8888ebf-b4d4-44ab-a54f-64c07e6b13e4 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:09.687751 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8888ebf-b4d4-44ab-a54f-64c07e6b13e4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:09.687751 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8888ebf-b4d4-44ab-a54f-64c07e6b13e4 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:09.689080 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 657/1332] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:09.694131 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f09139-7c22-4684-a747-ce4f2322f9ed None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:09.694578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f09139-7c22-4684-a747-ce4f2322f9ed None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:09.694719 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f09139-7c22-4684-a747-ce4f2322f9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:09.773241 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-89f09139-7c22-4684-a747-ce4f2322f9ed None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:09.774527 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-89f09139-7c22-4684-a747-ce4f2322f9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:09.871232 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 676/1333] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:23:10.160429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8888ebf-b4d4-44ab-a54f-64c07e6b13e4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:10.161959 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d8888ebf-b4d4-44ab-a54f-64c07e6b13e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:10.229221 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:10.262757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8888ebf-b4d4-44ab-a54f-64c07e6b13e4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:10.334389 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb5cd7-7c09-4352-8aa8-8ae34d77c683 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:10.334936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb5cd7-7c09-4352-8aa8-8ae34d77c683 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:10.335244 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb5cd7-7c09-4352-8aa8-8ae34d77c683 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:10.335519 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb5cd7-7c09-4352-8aa8-8ae34d77c683 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:10.337165 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-18eb5cd7-7c09-4352-8aa8-8ae34d77c683 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:10.457391 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb5cd7-7c09-4352-8aa8-8ae34d77c683 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:10.458878 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb5cd7-7c09-4352-8aa8-8ae34d77c683 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:10.469103 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:10.478027 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 658/1334] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:23:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:23:12.429762 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e40ae13-0df3-49a1-9400-37c82d91d74e None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:12.430300 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e40ae13-0df3-49a1-9400-37c82d91d74e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:12.430498 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e40ae13-0df3-49a1-9400-37c82d91d74e None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:12.431663 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 677/1335] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:12.437529 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d41b10-1cf6-4ca5-a622-f079a3e84471 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:12.437952 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d41b10-1cf6-4ca5-a622-f079a3e84471 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:12.438073 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d41b10-1cf6-4ca5-a622-f079a3e84471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:12.507760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-85d41b10-1cf6-4ca5-a622-f079a3e84471 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:12.509143 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-85d41b10-1cf6-4ca5-a622-f079a3e84471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:12.604288 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 659/1336] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:23:12.894952 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e40ae13-0df3-49a1-9400-37c82d91d74e None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:12.896536 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2e40ae13-0df3-49a1-9400-37c82d91d74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:12.935648 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:12.999868 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e40ae13-0df3-49a1-9400-37c82d91d74e None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:13.074230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf07e73f-f088-4d78-9fb9-8fa6272f4003 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:13.074838 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07e73f-f088-4d78-9fb9-8fa6272f4003 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:13.075114 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07e73f-f088-4d78-9fb9-8fa6272f4003 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:13.075383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07e73f-f088-4d78-9fb9-8fa6272f4003 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:13.076991 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bf07e73f-f088-4d78-9fb9-8fa6272f4003 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:13.158495 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:13.184713 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf07e73f-f088-4d78-9fb9-8fa6272f4003 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:13.186145 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf07e73f-f088-4d78-9fb9-8fa6272f4003 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:13.204406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 678/1337] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:23:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:23:15.449698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aff153-ab6d-4bc2-9066-0c705f8bf378 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:15.450247 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aff153-ab6d-4bc2-9066-0c705f8bf378 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:15.450359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aff153-ab6d-4bc2-9066-0c705f8bf378 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:15.451615 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 660/1338] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:15.457244 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb94e5d-9012-47b1-8619-47ce104223e9 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:15.457673 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb94e5d-9012-47b1-8619-47ce104223e9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:15.457798 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb94e5d-9012-47b1-8619-47ce104223e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:15.523237 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5eb94e5d-9012-47b1-8619-47ce104223e9 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:15.524452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5eb94e5d-9012-47b1-8619-47ce104223e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:15.619051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 679/1339] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:23:15.904153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aff153-ab6d-4bc2-9066-0c705f8bf378 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:15.905739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-88aff153-ab6d-4bc2-9066-0c705f8bf378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:15.980286 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:16.007592 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aff153-ab6d-4bc2-9066-0c705f8bf378 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:16.079876 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631a2fc-abd1-4bb1-97ec-b340a68e71f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:16.080981 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631a2fc-abd1-4bb1-97ec-b340a68e71f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:16.081262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631a2fc-abd1-4bb1-97ec-b340a68e71f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:16.081531 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631a2fc-abd1-4bb1-97ec-b340a68e71f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:16.083165 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c631a2fc-abd1-4bb1-97ec-b340a68e71f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:16.192169 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c631a2fc-abd1-4bb1-97ec-b340a68e71f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:16.193606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c631a2fc-abd1-4bb1-97ec-b340a68e71f8 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:16.211589 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 661/1340] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:23:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:23:20.149496 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde17bb9-48b7-47a8-af5f-0484e1316f9b None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:20.149920 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde17bb9-48b7-47a8-af5f-0484e1316f9b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:20.150215 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde17bb9-48b7-47a8-af5f-0484e1316f9b None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:20.151654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 680/1341] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:20.157242 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c92e4-5b18-4f91-a98d-f77bc5fb1435 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:20.157678 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c92e4-5b18-4f91-a98d-f77bc5fb1435 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:20.157839 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c92e4-5b18-4f91-a98d-f77bc5fb1435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:20.163351 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:20.225993 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-271c92e4-5b18-4f91-a98d-f77bc5fb1435 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:20.227228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-271c92e4-5b18-4f91-a98d-f77bc5fb1435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:20.322060 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 662/1342] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:23:20.605629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde17bb9-48b7-47a8-af5f-0484e1316f9b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:20.607261 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-cde17bb9-48b7-47a8-af5f-0484e1316f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:20.655823 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:20.713140 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde17bb9-48b7-47a8-af5f-0484e1316f9b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:20.787410 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0eff20-d89c-4ee1-a579-f8ed4d433f52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:20.788016 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0eff20-d89c-4ee1-a579-f8ed4d433f52 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:20.788296 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0eff20-d89c-4ee1-a579-f8ed4d433f52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:20.788571 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0eff20-d89c-4ee1-a579-f8ed4d433f52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:20.790158 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-dc0eff20-d89c-4ee1-a579-f8ed4d433f52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:20.882779 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:20.899343 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0eff20-d89c-4ee1-a579-f8ed4d433f52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:20.900763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0eff20-d89c-4ee1-a579-f8ed4d433f52 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:20.918556 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 681/1343] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:23:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:23:23.430740 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df03eb-d51e-4b37-89aa-c47c835b88ed None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:23.431315 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df03eb-d51e-4b37-89aa-c47c835b88ed None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:23.431315 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df03eb-d51e-4b37-89aa-c47c835b88ed None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:23.432646 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 663/1344] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:23.438331 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e25c2a-84a8-4a9d-8b15-25652b4ad581 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:23.439371 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e25c2a-84a8-4a9d-8b15-25652b4ad581 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:23.439451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e25c2a-84a8-4a9d-8b15-25652b4ad581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:23.513654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-89e25c2a-84a8-4a9d-8b15-25652b4ad581 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:23.515140 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-89e25c2a-84a8-4a9d-8b15-25652b4ad581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:23.615517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 682/1345] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:23:23.895932 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df03eb-d51e-4b37-89aa-c47c835b88ed None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:23.897461 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d9df03eb-d51e-4b37-89aa-c47c835b88ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:23.983389 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:24.001419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df03eb-d51e-4b37-89aa-c47c835b88ed None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:24.074761 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d5920-44b2-4a91-83dd-840ddec9e29e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:24.075370 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d5920-44b2-4a91-83dd-840ddec9e29e service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:24.075650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d5920-44b2-4a91-83dd-840ddec9e29e service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:24.075921 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d5920-44b2-4a91-83dd-840ddec9e29e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:24.077544 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fe0d5920-44b2-4a91-83dd-840ddec9e29e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:24.187330 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d5920-44b2-4a91-83dd-840ddec9e29e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:24.188813 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d5920-44b2-4a91-83dd-840ddec9e29e service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:24.208524 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 664/1346] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:23:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:23:28.175891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44750955-23a1-467f-a956-bf849bd7cdf9 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:28.176498 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44750955-23a1-467f-a956-bf849bd7cdf9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:28.176498 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44750955-23a1-467f-a956-bf849bd7cdf9 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:28.177766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 683/1347] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:28.183296 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775dcd4-60eb-40e6-884d-d535ad4862e2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:28.183704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775dcd4-60eb-40e6-884d-d535ad4862e2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:28.183985 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775dcd4-60eb-40e6-884d-d535ad4862e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:28.221641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:28.279061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d775dcd4-60eb-40e6-884d-d535ad4862e2 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:28.280293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d775dcd4-60eb-40e6-884d-d535ad4862e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:28.408737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 665/1348] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:23:28.689908 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44750955-23a1-467f-a956-bf849bd7cdf9 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:28.691543 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-44750955-23a1-467f-a956-bf849bd7cdf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:28.761407 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:28.809071 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44750955-23a1-467f-a956-bf849bd7cdf9 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:28.885729 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32affbd0-78aa-43c0-9203-53c7d5211159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:28.886383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32affbd0-78aa-43c0-9203-53c7d5211159 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:28.886666 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32affbd0-78aa-43c0-9203-53c7d5211159 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:28.886937 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32affbd0-78aa-43c0-9203-53c7d5211159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:28.888535 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-32affbd0-78aa-43c0-9203-53c7d5211159 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:28.992992 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:28.997288 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32affbd0-78aa-43c0-9203-53c7d5211159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:28.998763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32affbd0-78aa-43c0-9203-53c7d5211159 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:29.016783 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 684/1349] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:23:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:23:33.931035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9ec55-43c6-416c-bb31-530f053bde26 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:33.931602 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9ec55-43c6-416c-bb31-530f053bde26 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:33.931602 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9ec55-43c6-416c-bb31-530f053bde26 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:33.934120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 666/1350] 10.222.0.58 () {54 vars in 921 bytes} [Thu Sep 1 16:23:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:33.940378 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034807f-44f1-4443-ac3b-11301a5a4e04 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:33.940810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034807f-44f1-4443-ac3b-11301a5a4e04 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:33.941598 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034807f-44f1-4443-ac3b-11301a5a4e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:34.051758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-6034807f-44f1-4443-ac3b-11301a5a4e04 None None] MFA Rules not processed for user `e12302cdb2854b0684179647a6dadf6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:34.053330 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6034807f-44f1-4443-ac3b-11301a5a4e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:34.214259 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 685/1351] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Sep 1 16:23:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:23:38.406817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=59 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:38.502245 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858ce35-7be7-4f58-86ce-cee9c8bb95dc None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:38.502617 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858ce35-7be7-4f58-86ce-cee9c8bb95dc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:38.502873 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858ce35-7be7-4f58-86ce-cee9c8bb95dc None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:38.504320 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 667/1352] 10.222.0.58 () {54 vars in 921 bytes} [Thu Sep 1 16:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:38.509231 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d989196-b8ca-4c6b-a845-495ce934e8c9 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:38.510068 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d989196-b8ca-4c6b-a845-495ce934e8c9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:38.510667 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d989196-b8ca-4c6b-a845-495ce934e8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:38.571607 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:38.597654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2d989196-b8ca-4c6b-a845-495ce934e8c9 None None] MFA Rules not processed for user `e12302cdb2854b0684179647a6dadf6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:38.599250 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2d989196-b8ca-4c6b-a845-495ce934e8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:38.720519 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 686/1353] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Sep 1 16:23:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:23:40.028829 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e95910-279a-43b8-ab56-78a4a6c3b249 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:40.029377 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e95910-279a-43b8-ab56-78a4a6c3b249 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:40.029377 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e95910-279a-43b8-ab56-78a4a6c3b249 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:40.031615 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 668/1354] 10.222.0.58 () {54 vars in 921 bytes} [Thu Sep 1 16:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:40.036810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb72e3b-304f-4fef-a395-0ccfee8f4c97 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:40.037274 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb72e3b-304f-4fef-a395-0ccfee8f4c97 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:40.037388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb72e3b-304f-4fef-a395-0ccfee8f4c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:40.116965 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-fcb72e3b-304f-4fef-a395-0ccfee8f4c97 None None] MFA Rules not processed for user `e12302cdb2854b0684179647a6dadf6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:40.118164 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-fcb72e3b-304f-4fef-a395-0ccfee8f4c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:40.160829 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:40.225412 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 687/1355] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Sep 1 16:23:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:23:43.545974 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d1764-296f-4534-94e8-6073644877e3 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:43.546548 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d1764-296f-4534-94e8-6073644877e3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:43.546648 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d1764-296f-4534-94e8-6073644877e3 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:43.548083 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 669/1356] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:43.553569 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa0d87-49a5-49d6-8ade-fba58a18100f None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:43.553976 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa0d87-49a5-49d6-8ade-fba58a18100f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:43.554351 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa0d87-49a5-49d6-8ade-fba58a18100f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:43.648911 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-1bfa0d87-49a5-49d6-8ade-fba58a18100f None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:43.650328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1bfa0d87-49a5-49d6-8ade-fba58a18100f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:43.776355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 688/1357] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:23:44.075866 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6c79e-7129-4133-82f7-8e233d5bb20f None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:44.076580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6c79e-7129-4133-82f7-8e233d5bb20f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:44.077077 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6c79e-7129-4133-82f7-8e233d5bb20f None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:44.079778 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 670/1358] 10.222.0.58 () {54 vars in 962 bytes} [Thu Sep 1 16:23:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:44.095638 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce9111-d0bc-4685-8b9c-ca7baaa8ed12 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:44.097212 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce9111-d0bc-4685-8b9c-ca7baaa8ed12 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:44.098379 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce9111-d0bc-4685-8b9c-ca7baaa8ed12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:44.119200 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:44.193953 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-38ce9111-d0bc-4685-8b9c-ca7baaa8ed12 None None] MFA Rules not processed for user `1f0c99fdf8bd4331ad7e641ee38554c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:44.195357 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-38ce9111-d0bc-4685-8b9c-ca7baaa8ed12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:44.322470 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 689/1359] 10.222.0.58 () {58 vars in 1059 bytes} [Thu Sep 1 16:23:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:23:44.331035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6c79e-7129-4133-82f7-8e233d5bb20f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:44.332644 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-41a6c79e-7129-4133-82f7-8e233d5bb20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:44.492024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6c79e-7129-4133-82f7-8e233d5bb20f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:44.573108 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:44.618994 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956900bc-3bce-43c1-82fd-52a534e4b3e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:44.619782 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956900bc-3bce-43c1-82fd-52a534e4b3e1 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:44.620168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956900bc-3bce-43c1-82fd-52a534e4b3e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:44.620543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956900bc-3bce-43c1-82fd-52a534e4b3e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:44.622281 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-956900bc-3bce-43c1-82fd-52a534e4b3e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:44.715935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956900bc-3bce-43c1-82fd-52a534e4b3e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:44.717554 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956900bc-3bce-43c1-82fd-52a534e4b3e1 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:44.737060 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 671/1360] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:23:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:23:46.872086 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f22c4c-f713-4233-b76a-dcb64216eb52 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:46.872643 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f22c4c-f713-4233-b76a-dcb64216eb52 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:46.872643 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f22c4c-f713-4233-b76a-dcb64216eb52 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:46.873935 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 690/1361] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:46.879321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee05d49-c88a-4398-976d-d5d90307d03d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:46.879745 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee05d49-c88a-4398-976d-d5d90307d03d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:46.879861 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee05d49-c88a-4398-976d-d5d90307d03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:46.949429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-aee05d49-c88a-4398-976d-d5d90307d03d None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:46.950806 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-aee05d49-c88a-4398-976d-d5d90307d03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:46.989948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:47.061684 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 672/1362] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:23:47.350932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f22c4c-f713-4233-b76a-dcb64216eb52 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:47.352303 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-84f22c4c-f713-4233-b76a-dcb64216eb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:47.421676 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:47.508140 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f22c4c-f713-4233-b76a-dcb64216eb52 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:47.625602 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1c510-c049-46a2-9838-15b6880f5a16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:47.626117 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1c510-c049-46a2-9838-15b6880f5a16 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:47.626421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1c510-c049-46a2-9838-15b6880f5a16 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:47.626692 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1c510-c049-46a2-9838-15b6880f5a16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:47.628311 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-08f1c510-c049-46a2-9838-15b6880f5a16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:47.651612 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:47.729562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1c510-c049-46a2-9838-15b6880f5a16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:47.731270 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1c510-c049-46a2-9838-15b6880f5a16 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:47.750316 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 691/1363] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:23:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:23:50.044915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f8641-0464-4d57-beb8-46ace47fcc33 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:50.044915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f8641-0464-4d57-beb8-46ace47fcc33 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:50.044915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f8641-0464-4d57-beb8-46ace47fcc33 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:50.046660 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 673/1364] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:23:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:50.052153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b3b8e-8675-44fa-8156-47a8db0d3b7a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:50.052555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b3b8e-8675-44fa-8156-47a8db0d3b7a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:50.052734 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b3b8e-8675-44fa-8156-47a8db0d3b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:50.125159 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ab6b3b8e-8675-44fa-8156-47a8db0d3b7a None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:50.126412 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ab6b3b8e-8675-44fa-8156-47a8db0d3b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:50.235772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:50.239278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 692/1365] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:23:50.542077 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af6d4e-04a8-43f4-a821-f1089c9b5d5f None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:50.542722 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af6d4e-04a8-43f4-a821-f1089c9b5d5f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:50.543222 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af6d4e-04a8-43f4-a821-f1089c9b5d5f None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:50.544749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 674/1366] 10.222.0.58 () {54 vars in 962 bytes} [Thu Sep 1 16:23:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:50.550930 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2b2de-62f3-44d4-93bc-03bb78ffb66b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:50.551282 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2b2de-62f3-44d4-93bc-03bb78ffb66b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:50.551621 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2b2de-62f3-44d4-93bc-03bb78ffb66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:50.634098 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d4d2b2de-62f3-44d4-93bc-03bb78ffb66b None None] MFA Rules not processed for user `1f0c99fdf8bd4331ad7e641ee38554c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:50.635344 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d4d2b2de-62f3-44d4-93bc-03bb78ffb66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:50.769680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 693/1367] 10.222.0.58 () {58 vars in 1059 bytes} [Thu Sep 1 16:23:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:23:50.777758 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af6d4e-04a8-43f4-a821-f1089c9b5d5f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:50.779816 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-65af6d4e-04a8-43f4-a821-f1089c9b5d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:50.946810 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af6d4e-04a8-43f4-a821-f1089c9b5d5f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:50.969380 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:51.076066 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24bd4b6-bbc3-49b2-8b74-6e02376976b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:51.076679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24bd4b6-bbc3-49b2-8b74-6e02376976b1 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:51.077100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24bd4b6-bbc3-49b2-8b74-6e02376976b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:51.077377 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24bd4b6-bbc3-49b2-8b74-6e02376976b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:51.079421 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e24bd4b6-bbc3-49b2-8b74-6e02376976b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:51.185521 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24bd4b6-bbc3-49b2-8b74-6e02376976b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:51.187364 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24bd4b6-bbc3-49b2-8b74-6e02376976b1 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:51.206321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 675/1368] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:23:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:23:51.735231 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c691bf-2bbb-4624-96d3-403fafa778e4 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:51.735775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c691bf-2bbb-4624-96d3-403fafa778e4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:51.735775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c691bf-2bbb-4624-96d3-403fafa778e4 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:51.737245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 694/1369] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:51.742359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fe611-2b49-4f7c-86e7-0bd8f26b6078 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:51.742583 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fe611-2b49-4f7c-86e7-0bd8f26b6078 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:51.742846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fe611-2b49-4f7c-86e7-0bd8f26b6078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:51.778381 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:51.817443 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-645fe611-2b49-4f7c-86e7-0bd8f26b6078 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:51.818779 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-645fe611-2b49-4f7c-86e7-0bd8f26b6078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:51.946436 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 676/1370] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:23:52.543538 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdd750-013d-499a-8cf1-3c054cf1172a None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:52.543890 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdd750-013d-499a-8cf1-3c054cf1172a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:52.544236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdd750-013d-499a-8cf1-3c054cf1172a None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:52.545877 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 695/1371] 10.222.0.58 () {54 vars in 964 bytes} [Thu Sep 1 16:23:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:52.552594 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d42f068-4a6c-4e47-a161-bbee03d7b456 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:52.553102 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d42f068-4a6c-4e47-a161-bbee03d7b456 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:52.553207 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d42f068-4a6c-4e47-a161-bbee03d7b456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:52.649316 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0d42f068-4a6c-4e47-a161-bbee03d7b456 None None] MFA Rules not processed for user `84d4bd0a456341f78b5e476c97a9165c`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:52.650537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0d42f068-4a6c-4e47-a161-bbee03d7b456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:52.699105 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:52.731574 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 677/1372] 10.222.0.58 () {58 vars in 1061 bytes} [Thu Sep 1 16:23:52 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:23:52.739731 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdd750-013d-499a-8cf1-3c054cf1172a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:52.741226 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-83cdd750-013d-499a-8cf1-3c054cf1172a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:52.794392 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:52.857105 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdd750-013d-499a-8cf1-3c054cf1172a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:52.941683 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6fa9b7-45b7-44e2-bcca-776b7ada6559 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:52.942305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fa9b7-45b7-44e2-bcca-776b7ada6559 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:52.942644 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fa9b7-45b7-44e2-bcca-776b7ada6559 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:52.942961 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fa9b7-45b7-44e2-bcca-776b7ada6559 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:52.944388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8e6fa9b7-45b7-44e2-bcca-776b7ada6559 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:53.052472 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:53.072142 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6fa9b7-45b7-44e2-bcca-776b7ada6559 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:53.072142 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6fa9b7-45b7-44e2-bcca-776b7ada6559 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:53.105886 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 696/1373] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:23:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:23:53.213437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d906ab7-3f9e-47d7-ae42-7c3ecd5635b9 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:53.213727 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d906ab7-3f9e-47d7-ae42-7c3ecd5635b9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:53.214035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d906ab7-3f9e-47d7-ae42-7c3ecd5635b9 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:53.215550 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 678/1374] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:53.221898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e55be-7002-4e8f-8326-82896248015d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:53.221898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e55be-7002-4e8f-8326-82896248015d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:53.221898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e55be-7002-4e8f-8326-82896248015d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:53.293933 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e39e55be-7002-4e8f-8326-82896248015d None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:53.295501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e39e55be-7002-4e8f-8326-82896248015d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:53.394745 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 697/1375] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:23:53.682958 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d906ab7-3f9e-47d7-ae42-7c3ecd5635b9 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:53.684481 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2d906ab7-3f9e-47d7-ae42-7c3ecd5635b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:53.839469 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d906ab7-3f9e-47d7-ae42-7c3ecd5635b9 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:53.894049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:53.954789 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbea837-e86d-4da8-ac5a-7e5f1038f258 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:53.955315 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbea837-e86d-4da8-ac5a-7e5f1038f258 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:53.955592 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbea837-e86d-4da8-ac5a-7e5f1038f258 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:53.955861 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbea837-e86d-4da8-ac5a-7e5f1038f258 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:53.957447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bbbea837-e86d-4da8-ac5a-7e5f1038f258 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:54.048842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbea837-e86d-4da8-ac5a-7e5f1038f258 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:54.050505 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbea837-e86d-4da8-ac5a-7e5f1038f258 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:54.072627 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 679/1376] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:23:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:23:56.015677 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca2ac6-0f8b-4b7e-b5da-744a1b2e0e98 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:56.016148 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca2ac6-0f8b-4b7e-b5da-744a1b2e0e98 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:56.016244 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca2ac6-0f8b-4b7e-b5da-744a1b2e0e98 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:56.017876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 698/1377] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:23:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:56.023286 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e425e-ac65-457d-840f-767756ee312e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:56.023691 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e425e-ac65-457d-840f-767756ee312e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:56.023855 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e425e-ac65-457d-840f-767756ee312e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:56.109405 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-539e425e-ac65-457d-840f-767756ee312e None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:56.110939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-539e425e-ac65-457d-840f-767756ee312e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:56.119202 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:56.212881 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 680/1378] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:23:56.503114 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ca2ac6-0f8b-4b7e-b5da-744a1b2e0e98 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:56.504629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-09ca2ac6-0f8b-4b7e-b5da-744a1b2e0e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:56.600104 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:56.698966 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ca2ac6-0f8b-4b7e-b5da-744a1b2e0e98 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:56.817920 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b961eda-9ac6-47fb-ad19-56a94203b7bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:56.818564 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b961eda-9ac6-47fb-ad19-56a94203b7bf service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:56.818836 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b961eda-9ac6-47fb-ad19-56a94203b7bf service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:56.819101 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b961eda-9ac6-47fb-ad19-56a94203b7bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:56.820947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8b961eda-9ac6-47fb-ad19-56a94203b7bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:56.840653 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:56.916879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b961eda-9ac6-47fb-ad19-56a94203b7bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:56.918533 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b961eda-9ac6-47fb-ad19-56a94203b7bf service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:56.937576 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 699/1379] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:23:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:23:58.892571 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428ba44-7b36-429c-bd38-8a54f6502b6d None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:58.892571 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428ba44-7b36-429c-bd38-8a54f6502b6d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:58.892571 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428ba44-7b36-429c-bd38-8a54f6502b6d None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:58.899885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbd1a5-71e0-4c19-a5cd-30a332814950 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:58.899885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbd1a5-71e0-4c19-a5cd-30a332814950 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:58.899885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbd1a5-71e0-4c19-a5cd-30a332814950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:58.904387 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 681/1380] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:23:58 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:58.968646 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-28dbd1a5-71e0-4c19-a5cd-30a332814950 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:58.969850 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-28dbd1a5-71e0-4c19-a5cd-30a332814950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:59.067738 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:59.072187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 700/1381] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:23:59.363096 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e428ba44-7b36-429c-bd38-8a54f6502b6d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:59.367168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e428ba44-7b36-429c-bd38-8a54f6502b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:59.539436 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:59.551227 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e428ba44-7b36-429c-bd38-8a54f6502b6d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:59.675152 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413739ef-fd29-4125-8463-cf79c1fa7c53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:59.675752 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413739ef-fd29-4125-8463-cf79c1fa7c53 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:59.676040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413739ef-fd29-4125-8463-cf79c1fa7c53 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:59.676318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413739ef-fd29-4125-8463-cf79c1fa7c53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:59.678022 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-413739ef-fd29-4125-8463-cf79c1fa7c53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:59.788868 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413739ef-fd29-4125-8463-cf79c1fa7c53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:59.790503 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413739ef-fd29-4125-8463-cf79c1fa7c53 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:59.802173 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:59.812374 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 682/1382] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:23:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:24:01.774885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c679ee7-3e77-4fb5-a2c9-fc6a13674162 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:01.775462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c679ee7-3e77-4fb5-a2c9-fc6a13674162 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:01.775564 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c679ee7-3e77-4fb5-a2c9-fc6a13674162 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:01.777250 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 701/1383] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:24:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:01.785005 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aac84f-a0a2-42fd-8ad7-f4cdcffc9da7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:01.785407 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aac84f-a0a2-42fd-8ad7-f4cdcffc9da7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:01.785638 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aac84f-a0a2-42fd-8ad7-f4cdcffc9da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:01.862872 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d8aac84f-a0a2-42fd-8ad7-f4cdcffc9da7 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:01.864265 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d8aac84f-a0a2-42fd-8ad7-f4cdcffc9da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:01.976208 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 683/1384] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:02.307995 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c679ee7-3e77-4fb5-a2c9-fc6a13674162 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:02.309472 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3c679ee7-3e77-4fb5-a2c9-fc6a13674162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:02.475865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c679ee7-3e77-4fb5-a2c9-fc6a13674162 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:02.555023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:02.609267 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896573e-4ced-494b-a108-3968ecaffdd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:02.609910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896573e-4ced-494b-a108-3968ecaffdd7 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:02.610243 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896573e-4ced-494b-a108-3968ecaffdd7 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:02.610583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896573e-4ced-494b-a108-3968ecaffdd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:02.612398 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a896573e-4ced-494b-a108-3968ecaffdd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:02.714644 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a896573e-4ced-494b-a108-3968ecaffdd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:02.716244 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a896573e-4ced-494b-a108-3968ecaffdd7 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:02.735085 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 702/1385] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:24:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:24:04.681268 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27a87a-b96f-46ac-a54c-32c41ca333eb None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:04.681896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27a87a-b96f-46ac-a54c-32c41ca333eb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:04.682100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27a87a-b96f-46ac-a54c-32c41ca333eb None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:04.683886 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 684/1386] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:24:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:04.691168 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68a981-0745-4c8d-9d39-7c1192481ad9 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:04.691626 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68a981-0745-4c8d-9d39-7c1192481ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:04.691838 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68a981-0745-4c8d-9d39-7c1192481ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:04.764352 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-6c68a981-0745-4c8d-9d39-7c1192481ad9 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:04.765603 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6c68a981-0745-4c8d-9d39-7c1192481ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:04.806251 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:04.885525 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 703/1387] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:05.259853 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27a87a-b96f-46ac-a54c-32c41ca333eb None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:05.261875 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cc27a87a-b96f-46ac-a54c-32c41ca333eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:05.354852 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:05.435530 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27a87a-b96f-46ac-a54c-32c41ca333eb None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:05.561430 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1904115-43c7-4401-be78-7a3e459c71e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:05.561964 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1904115-43c7-4401-be78-7a3e459c71e6 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:05.562332 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1904115-43c7-4401-be78-7a3e459c71e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:05.562576 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1904115-43c7-4401-be78-7a3e459c71e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:05.564294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f1904115-43c7-4401-be78-7a3e459c71e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:05.605273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:05.664450 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1904115-43c7-4401-be78-7a3e459c71e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:05.666078 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1904115-43c7-4401-be78-7a3e459c71e6 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:05.684415 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 685/1388] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:24:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:24:07.671283 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ef126-cb7e-4eb0-9fdd-4bb9ce9fe5f7 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:07.671841 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ef126-cb7e-4eb0-9fdd-4bb9ce9fe5f7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:07.671841 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ef126-cb7e-4eb0-9fdd-4bb9ce9fe5f7 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:07.673340 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 704/1389] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:07.678481 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79427d19-4760-4502-be00-62b546c64ef6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:07.678896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79427d19-4760-4502-be00-62b546c64ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:07.679143 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79427d19-4760-4502-be00-62b546c64ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:07.744343 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-79427d19-4760-4502-be00-62b546c64ef6 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:07.745541 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-79427d19-4760-4502-be00-62b546c64ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:07.846991 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 686/1390] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:08.140128 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ef126-cb7e-4eb0-9fdd-4bb9ce9fe5f7 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:08.141707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b48ef126-cb7e-4eb0-9fdd-4bb9ce9fe5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:08.234390 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab23bd3-ed79-49a9-924e-382556855878 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:08.234744 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab23bd3-ed79-49a9-924e-382556855878 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:08.234943 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab23bd3-ed79-49a9-924e-382556855878 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:08.236314 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 687/1391] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:24:08.241494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bfdfb-3860-4212-9c5c-3348130eef58 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:08.241773 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bfdfb-3860-4212-9c5c-3348130eef58 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:08.242045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bfdfb-3860-4212-9c5c-3348130eef58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:08.257675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:08.290349 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ef126-cb7e-4eb0-9fdd-4bb9ce9fe5f7 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:08.306496 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:08.311970 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7c9bfdfb-3860-4212-9c5c-3348130eef58 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:08.313190 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7c9bfdfb-3860-4212-9c5c-3348130eef58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:08.405453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f773cf4-20c7-403d-a04a-c47652ef1a3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:08.406065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f773cf4-20c7-403d-a04a-c47652ef1a3a service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:08.406376 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f773cf4-20c7-403d-a04a-c47652ef1a3a service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:08.406652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f773cf4-20c7-403d-a04a-c47652ef1a3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:08.408292 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2f773cf4-20c7-403d-a04a-c47652ef1a3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:08.413450 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 688/1392] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:24:08.499603 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f773cf4-20c7-403d-a04a-c47652ef1a3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:08.501252 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f773cf4-20c7-403d-a04a-c47652ef1a3a service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:08.519951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 705/1393] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:24:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:24:08.714619 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43e584-c785-4194-b7c8-183b4205144f None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:08.715133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43e584-c785-4194-b7c8-183b4205144f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:08.715253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43e584-c785-4194-b7c8-183b4205144f None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:08.717502 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 689/1394] 10.222.0.58 () {54 vars in 967 bytes} [Thu Sep 1 16:24:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:08.723639 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ee5c7-bc2e-475b-ae17-40a98c173a99 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:08.724099 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ee5c7-bc2e-475b-ae17-40a98c173a99 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:08.724368 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ee5c7-bc2e-475b-ae17-40a98c173a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:08.750168 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:08.802460 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-618ee5c7-bc2e-475b-ae17-40a98c173a99 None None] MFA Rules not processed for user `6a2c872129414105a112661cc2b477e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:08.803677 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-618ee5c7-bc2e-475b-ae17-40a98c173a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:08.878795 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 706/1395] 10.222.0.58 () {58 vars in 1064 bytes} [Thu Sep 1 16:24:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:08.886187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43e584-c785-4194-b7c8-183b4205144f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:08.888288 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1b43e584-c785-4194-b7c8-183b4205144f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:08.986473 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:08.996462 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43e584-c785-4194-b7c8-183b4205144f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:09.074500 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4902d-a183-4afa-8210-69c7bbd99c05 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2c872129414105a112661cc2b477e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:09.075109 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4902d-a183-4afa-8210-69c7bbd99c05 service glance] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:09.075392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4902d-a183-4afa-8210-69c7bbd99c05 service glance] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:09.075673 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4902d-a183-4afa-8210-69c7bbd99c05 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:09.077274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f1e4902d-a183-4afa-8210-69c7bbd99c05 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:09.185334 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e4902d-a183-4afa-8210-69c7bbd99c05 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:09.186839 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e4902d-a183-4afa-8210-69c7bbd99c05 service glance] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:09.204897 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 690/1396] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:24:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:24:09.975655 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cd529-4b7c-46a6-b341-55b4e85f72f3 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:09.976157 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cd529-4b7c-46a6-b341-55b4e85f72f3 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:09.976340 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cd529-4b7c-46a6-b341-55b4e85f72f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:10.046754 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e04cd529-4b7c-46a6-b341-55b4e85f72f3 None None] MFA Rules not processed for user `dd7b98536fa445cebd552fcbf54b757d`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:10.047964 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e04cd529-4b7c-46a6-b341-55b4e85f72f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:10.107256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:10.135990 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 707/1397] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Sep 1 16:24:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:10.179138 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57a54d-2fe1-4d55-8e7b-7c9cc4bc6731 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:10.179632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57a54d-2fe1-4d55-8e7b-7c9cc4bc6731 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:10.179746 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57a54d-2fe1-4d55-8e7b-7c9cc4bc6731 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:10.181119 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 691/1398] 10.222.0.58 () {54 vars in 943 bytes} [Thu Sep 1 16:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:10.187487 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060973bf-b649-4168-844e-89376797edb9 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:10.187773 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060973bf-b649-4168-844e-89376797edb9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:10.188041 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060973bf-b649-4168-844e-89376797edb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:10.265333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-060973bf-b649-4168-844e-89376797edb9 None None] MFA Rules not processed for user `e789695517c1437fb224f5a6195655f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:10.267069 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-060973bf-b649-4168-844e-89376797edb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:10.372714 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 708/1399] 10.222.0.58 () {58 vars in 1040 bytes} [Thu Sep 1 16:24:10 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:10.382106 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57a54d-2fe1-4d55-8e7b-7c9cc4bc6731 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:10.383637 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fd57a54d-2fe1-4d55-8e7b-7c9cc4bc6731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:10.407303 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:10.476193 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42658f-9d15-4666-937a-d6c2ca52f5bd None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:10.476485 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42658f-9d15-4666-937a-d6c2ca52f5bd None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:10.476795 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42658f-9d15-4666-937a-d6c2ca52f5bd None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:10.478137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 709/1400] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:24:10.483227 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9086e-e8ed-413b-9a05-e7677c0b2e1c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:10.483627 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9086e-e8ed-413b-9a05-e7677c0b2e1c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:10.483908 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9086e-e8ed-413b-9a05-e7677c0b2e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:10.521913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57a54d-2fe1-4d55-8e7b-7c9cc4bc6731 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:10.530787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:10.552566 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d3d9086e-e8ed-413b-9a05-e7677c0b2e1c None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:10.553787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d3d9086e-e8ed-413b-9a05-e7677c0b2e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:10.628284 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f37d0-0ceb-49d9-9960-3d071e5c446e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e789695517c1437fb224f5a6195655f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:10.628892 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f37d0-0ceb-49d9-9960-3d071e5c446e service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:10.629176 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f37d0-0ceb-49d9-9960-3d071e5c446e service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:10.629451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f37d0-0ceb-49d9-9960-3d071e5c446e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:10.631130 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a51f37d0-0ceb-49d9-9960-3d071e5c446e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:10.638873 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:10.653686 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 710/1401] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:24:10.719205 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f37d0-0ceb-49d9-9960-3d071e5c446e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:10.720922 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f37d0-0ceb-49d9-9960-3d071e5c446e service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:10.728825 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 692/1402] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Sep 1 16:24:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:24:10.958875 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9086e-e8ed-413b-9a05-e7677c0b2e1c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:10.960123 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d3d9086e-e8ed-413b-9a05-e7677c0b2e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:11.094037 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9086e-e8ed-413b-9a05-e7677c0b2e1c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:11.097933 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:11.218179 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a2c1cb-f828-45de-a18b-ac03b72fcd7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:11.218800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2c1cb-f828-45de-a18b-ac03b72fcd7f service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:11.219075 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2c1cb-f828-45de-a18b-ac03b72fcd7f service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:11.219351 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2c1cb-f828-45de-a18b-ac03b72fcd7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:11.220996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e0a2c1cb-f828-45de-a18b-ac03b72fcd7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:11.321575 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a2c1cb-f828-45de-a18b-ac03b72fcd7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:11.323317 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a2c1cb-f828-45de-a18b-ac03b72fcd7f service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:11.344095 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 711/1403] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:24:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:24:13.305095 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab7fd1-acf3-404d-91d9-3202c2782cf0 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:13.305095 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab7fd1-acf3-404d-91d9-3202c2782cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:13.305095 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab7fd1-acf3-404d-91d9-3202c2782cf0 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:13.310162 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 693/1404] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:24:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:13.312886 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985833c0-3001-4f93-ba10-0b8aa6f27fda None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:13.313682 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985833c0-3001-4f93-ba10-0b8aa6f27fda None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:13.314323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985833c0-3001-4f93-ba10-0b8aa6f27fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:13.326355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:13.392232 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-985833c0-3001-4f93-ba10-0b8aa6f27fda None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:13.393656 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-985833c0-3001-4f93-ba10-0b8aa6f27fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:13.508037 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 712/1405] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:13.795012 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab7fd1-acf3-404d-91d9-3202c2782cf0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:13.796907 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-62ab7fd1-acf3-404d-91d9-3202c2782cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:13.978565 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab7fd1-acf3-404d-91d9-3202c2782cf0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:13.982538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:14.125959 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d7d30-e37e-4b4b-ab62-2d5a14b68fba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:14.126911 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d7d30-e37e-4b4b-ab62-2d5a14b68fba service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:14.127246 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d7d30-e37e-4b4b-ab62-2d5a14b68fba service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:14.127526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d7d30-e37e-4b4b-ab62-2d5a14b68fba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:14.129284 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-012d7d30-e37e-4b4b-ab62-2d5a14b68fba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:14.226460 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d7d30-e37e-4b4b-ab62-2d5a14b68fba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:14.228124 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d7d30-e37e-4b4b-ab62-2d5a14b68fba service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:14.248010 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 694/1406] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:24:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:24:16.267323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac56b6c-a109-433a-a72d-4e723da38bed None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:16.267905 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac56b6c-a109-433a-a72d-4e723da38bed None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:16.268359 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac56b6c-a109-433a-a72d-4e723da38bed None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:16.270297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 713/1407] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:24:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:16.275785 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24728beb-c150-46e0-8070-23f94305e594 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:16.278936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24728beb-c150-46e0-8070-23f94305e594 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:16.279447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24728beb-c150-46e0-8070-23f94305e594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:16.289432 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:16.354438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-24728beb-c150-46e0-8070-23f94305e594 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:16.355692 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-24728beb-c150-46e0-8070-23f94305e594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:16.462940 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 695/1408] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:16.759285 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac56b6c-a109-433a-a72d-4e723da38bed None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:16.760803 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8ac56b6c-a109-433a-a72d-4e723da38bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:16.856812 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:16.932457 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac56b6c-a109-433a-a72d-4e723da38bed None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:17.061925 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b16842-f982-4ab6-b8fc-f035cc459f4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:17.062574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b16842-f982-4ab6-b8fc-f035cc459f4e service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:17.062856 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b16842-f982-4ab6-b8fc-f035cc459f4e service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:17.063129 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b16842-f982-4ab6-b8fc-f035cc459f4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:17.064906 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-97b16842-f982-4ab6-b8fc-f035cc459f4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:17.122073 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:17.169252 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b16842-f982-4ab6-b8fc-f035cc459f4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:17.171056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b16842-f982-4ab6-b8fc-f035cc459f4e service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:17.189652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 714/1409] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:24:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:24:26.461246 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=62 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:27.186826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=23 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:33.034382 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a45f6-b448-4fa6-9c35-225b3d2dbf3c None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:33.034957 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a45f6-b448-4fa6-9c35-225b3d2dbf3c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:33.034957 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a45f6-b448-4fa6-9c35-225b3d2dbf3c None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:33.036314 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 696/1410] 10.222.0.58 () {54 vars in 918 bytes} [Thu Sep 1 16:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:33.041457 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1be271-85ad-4929-ac0f-de723ff689d2 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:33.042592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1be271-85ad-4929-ac0f-de723ff689d2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:33.042777 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1be271-85ad-4929-ac0f-de723ff689d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:33.122708 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ae1be271-85ad-4929-ac0f-de723ff689d2 None None] MFA Rules not processed for user `e12302cdb2854b0684179647a6dadf6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:33.123896 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ae1be271-85ad-4929-ac0f-de723ff689d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:33.229172 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 715/1411] 10.222.0.58 () {58 vars in 1015 bytes} [Thu Sep 1 16:24:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:33.244063 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ab029-7bf4-4d93-b32c-a208c5aac844 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:33.244349 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ab029-7bf4-4d93-b32c-a208c5aac844 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:33.244619 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ab029-7bf4-4d93-b32c-a208c5aac844 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:33.245979 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 697/1412] 10.222.0.58 () {54 vars in 918 bytes} [Thu Sep 1 16:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:33.250501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3f693-7a2f-435a-a050-5a221b25a3a0 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:33.250791 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3f693-7a2f-435a-a050-5a221b25a3a0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:33.251101 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3f693-7a2f-435a-a050-5a221b25a3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:33.322374 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:33.327141 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a5c3f693-7a2f-435a-a050-5a221b25a3a0 None None] MFA Rules not processed for user `1f0c99fdf8bd4331ad7e641ee38554c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:33.329172 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a5c3f693-7a2f-435a-a050-5a221b25a3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:33.443228 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 716/1413] 10.222.0.58 () {58 vars in 1015 bytes} [Thu Sep 1 16:24:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:35.780947 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5670773e-54df-43d0-9a12-c9f1cca69bc0 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:35.781483 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5670773e-54df-43d0-9a12-c9f1cca69bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:35.781483 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5670773e-54df-43d0-9a12-c9f1cca69bc0 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:35.782848 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 698/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:35.788113 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408358e-d56e-44e0-bc55-a43490c7b2fa None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:35.788519 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408358e-d56e-44e0-bc55-a43490c7b2fa None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:35.788663 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408358e-d56e-44e0-bc55-a43490c7b2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:35.855593 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7408358e-d56e-44e0-bc55-a43490c7b2fa None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:35.856802 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7408358e-d56e-44e0-bc55-a43490c7b2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:35.909855 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:35.951346 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 717/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:24:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:36.241766 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5670773e-54df-43d0-9a12-c9f1cca69bc0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:36.243397 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5670773e-54df-43d0-9a12-c9f1cca69bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:36.390348 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5670773e-54df-43d0-9a12-c9f1cca69bc0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:36.472747 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:36.512672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b125923-2a98-4a08-87d0-2097d078010f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:36.513204 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b125923-2a98-4a08-87d0-2097d078010f service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:36.513486 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b125923-2a98-4a08-87d0-2097d078010f service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:36.513761 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b125923-2a98-4a08-87d0-2097d078010f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:36.515448 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5b125923-2a98-4a08-87d0-2097d078010f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:36.603668 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b125923-2a98-4a08-87d0-2097d078010f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:36.605302 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b125923-2a98-4a08-87d0-2097d078010f service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:36.623547 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 699/1416] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:24:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:24:37.512319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc35e0-63da-472e-bcd7-8310f2966ff9 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:37.512911 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc35e0-63da-472e-bcd7-8310f2966ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:37.512911 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc35e0-63da-472e-bcd7-8310f2966ff9 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:37.514175 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 718/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:24:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:37.519037 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49d269-28f1-483e-8e58-c0a87b15a804 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:37.519488 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49d269-28f1-483e-8e58-c0a87b15a804 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:37.519602 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49d269-28f1-483e-8e58-c0a87b15a804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:37.593868 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-cf49d269-28f1-483e-8e58-c0a87b15a804 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:37.595016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cf49d269-28f1-483e-8e58-c0a87b15a804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:37.621417 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:37.691365 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 700/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:37.841082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc35e0-63da-472e-bcd7-8310f2966ff9 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:37.842664 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f4dc35e0-63da-472e-bcd7-8310f2966ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:37.987440 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc35e0-63da-472e-bcd7-8310f2966ff9 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:38.028373 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:38.102407 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec2f57-7b59-4b1c-8851-02b3abc546f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:38.103017 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec2f57-7b59-4b1c-8851-02b3abc546f7 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:38.103300 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec2f57-7b59-4b1c-8851-02b3abc546f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:38.103577 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec2f57-7b59-4b1c-8851-02b3abc546f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:38.105183 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-94ec2f57-7b59-4b1c-8851-02b3abc546f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:38.194330 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec2f57-7b59-4b1c-8851-02b3abc546f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:38.196363 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec2f57-7b59-4b1c-8851-02b3abc546f7 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:38.214556 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 719/1419] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:24:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:24:39.105987 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffed46-2cfb-4972-afb6-cd9d0e3ec58b None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:39.106547 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffed46-2cfb-4972-afb6-cd9d0e3ec58b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:39.106642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffed46-2cfb-4972-afb6-cd9d0e3ec58b None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:39.107912 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 701/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:24:39.112726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3098d-b3ff-4412-acf4-2e192de61818 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:39.113170 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3098d-b3ff-4412-acf4-2e192de61818 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:39.113281 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3098d-b3ff-4412-acf4-2e192de61818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:39.167708 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:39.202824 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2df3098d-b3ff-4412-acf4-2e192de61818 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:39.204072 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2df3098d-b3ff-4412-acf4-2e192de61818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:39.300396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 720/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:39.456095 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ffed46-2cfb-4972-afb6-cd9d0e3ec58b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:39.457591 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-59ffed46-2cfb-4972-afb6-cd9d0e3ec58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:39.601465 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ffed46-2cfb-4972-afb6-cd9d0e3ec58b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:39.614051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:39.716655 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e457a-535b-4fdc-b3dc-63859081bc80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:39.717255 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e457a-535b-4fdc-b3dc-63859081bc80 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:39.717533 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e457a-535b-4fdc-b3dc-63859081bc80 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:39.717802 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e457a-535b-4fdc-b3dc-63859081bc80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:39.719414 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6f6e457a-535b-4fdc-b3dc-63859081bc80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:39.805954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e457a-535b-4fdc-b3dc-63859081bc80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:39.807566 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e457a-535b-4fdc-b3dc-63859081bc80 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:39.825470 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 702/1422] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:24:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:24:40.695986 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cd3b0-7418-46e2-927e-0d037533f999 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:40.696561 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cd3b0-7418-46e2-927e-0d037533f999 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:40.696561 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cd3b0-7418-46e2-927e-0d037533f999 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:40.697873 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 721/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:40.703347 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3db9b-71de-40a5-a3ab-ba21e13c45d4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:40.703774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3db9b-71de-40a5-a3ab-ba21e13c45d4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:40.703900 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3db9b-71de-40a5-a3ab-ba21e13c45d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:40.723322 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:40.780661 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-70a3db9b-71de-40a5-a3ab-ba21e13c45d4 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:40.781860 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-70a3db9b-71de-40a5-a3ab-ba21e13c45d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:40.877081 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 703/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:41.027336 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cd3b0-7418-46e2-927e-0d037533f999 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:41.029051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f19cd3b0-7418-46e2-927e-0d037533f999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:41.138693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:41.175158 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cd3b0-7418-46e2-927e-0d037533f999 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:41.286594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c9cdc-929f-4206-8c03-28f8ddca8c04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:41.287123 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c9cdc-929f-4206-8c03-28f8ddca8c04 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:41.287405 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c9cdc-929f-4206-8c03-28f8ddca8c04 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:41.287677 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c9cdc-929f-4206-8c03-28f8ddca8c04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:41.289290 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-554c9cdc-929f-4206-8c03-28f8ddca8c04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:41.365965 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:41.379547 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c9cdc-929f-4206-8c03-28f8ddca8c04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:41.381150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c9cdc-929f-4206-8c03-28f8ddca8c04 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:41.399009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 722/1425] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:24:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:24:42.256584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f867d7a-3ea4-4359-9f09-15d10cae4a96 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:42.257144 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f867d7a-3ea4-4359-9f09-15d10cae4a96 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:42.257144 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f867d7a-3ea4-4359-9f09-15d10cae4a96 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:42.258505 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 704/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:42.263415 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530fb94-918b-4290-9d65-e25606621cc7 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:42.263842 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530fb94-918b-4290-9d65-e25606621cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:42.263964 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530fb94-918b-4290-9d65-e25606621cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:42.339089 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-c530fb94-918b-4290-9d65-e25606621cc7 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:42.340207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c530fb94-918b-4290-9d65-e25606621cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:42.433895 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 723/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:24:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:42.588578 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f867d7a-3ea4-4359-9f09-15d10cae4a96 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:42.590865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9f867d7a-3ea4-4359-9f09-15d10cae4a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:42.673928 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:42.736503 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f867d7a-3ea4-4359-9f09-15d10cae4a96 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:42.846436 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52bec7-513c-452b-bd43-ee1f5deec6fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:42.846966 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52bec7-513c-452b-bd43-ee1f5deec6fc service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:42.847247 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52bec7-513c-452b-bd43-ee1f5deec6fc service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:42.847518 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52bec7-513c-452b-bd43-ee1f5deec6fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:42.849109 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1e52bec7-513c-452b-bd43-ee1f5deec6fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:42.895902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:42.947677 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e52bec7-513c-452b-bd43-ee1f5deec6fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:42.949956 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e52bec7-513c-452b-bd43-ee1f5deec6fc service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:42.967653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 705/1428] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:24:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:24:43.783490 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da35d79-7e44-49e9-b901-b3d345784531 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:43.784035 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da35d79-7e44-49e9-b901-b3d345784531 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:43.784035 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da35d79-7e44-49e9-b901-b3d345784531 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:43.785384 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 724/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:43.790435 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6952dc6-d8bf-4901-be25-17b9b95a0742 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:43.790885 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6952dc6-d8bf-4901-be25-17b9b95a0742 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:43.791084 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6952dc6-d8bf-4901-be25-17b9b95a0742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:43.863538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d6952dc6-d8bf-4901-be25-17b9b95a0742 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:43.864748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d6952dc6-d8bf-4901-be25-17b9b95a0742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:43.959725 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 706/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:24:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:44.108387 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da35d79-7e44-49e9-b901-b3d345784531 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:44.109904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4da35d79-7e44-49e9-b901-b3d345784531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:44.158440 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:44.254302 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da35d79-7e44-49e9-b901-b3d345784531 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:44.366235 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a452f0-c16b-4382-a11f-688b460de7f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:44.366754 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a452f0-c16b-4382-a11f-688b460de7f0 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:44.367034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a452f0-c16b-4382-a11f-688b460de7f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:44.367303 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a452f0-c16b-4382-a11f-688b460de7f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:44.368900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-15a452f0-c16b-4382-a11f-688b460de7f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:44.379268 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:44.467569 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a452f0-c16b-4382-a11f-688b460de7f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:44.469176 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a452f0-c16b-4382-a11f-688b460de7f0 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:44.487243 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 725/1431] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:24:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:24:45.302446 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8896c5-fe5b-4af2-8e1b-b31af35b6d02 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:45.302996 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8896c5-fe5b-4af2-8e1b-b31af35b6d02 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:45.302996 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8896c5-fe5b-4af2-8e1b-b31af35b6d02 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:45.304341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 707/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:45.309953 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119ce2d-c94b-4c97-8ba5-cdf4f7894567 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:45.310424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119ce2d-c94b-4c97-8ba5-cdf4f7894567 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:45.310535 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119ce2d-c94b-4c97-8ba5-cdf4f7894567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:45.388491 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e119ce2d-c94b-4c97-8ba5-cdf4f7894567 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:45.389690 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e119ce2d-c94b-4c97-8ba5-cdf4f7894567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:45.457775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:45.487275 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 726/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:24:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:45.643408 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8896c5-fe5b-4af2-8e1b-b31af35b6d02 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:45.644960 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-da8896c5-fe5b-4af2-8e1b-b31af35b6d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:45.662945 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:45.789524 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8896c5-fe5b-4af2-8e1b-b31af35b6d02 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:45.874361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:45.900603 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77bcec-931f-476b-8df5-ab09101d8d42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:45.901206 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77bcec-931f-476b-8df5-ab09101d8d42 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:45.901513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77bcec-931f-476b-8df5-ab09101d8d42 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:45.901764 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77bcec-931f-476b-8df5-ab09101d8d42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:45.903679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0a77bcec-931f-476b-8df5-ab09101d8d42 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:45.993433 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77bcec-931f-476b-8df5-ab09101d8d42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:45.995127 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77bcec-931f-476b-8df5-ab09101d8d42 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:46.013910 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 708/1434] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:24:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:24:46.830737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed7cc6-f535-45b1-b73d-5da3d21cd466 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:46.831313 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed7cc6-f535-45b1-b73d-5da3d21cd466 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:46.831313 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed7cc6-f535-45b1-b73d-5da3d21cd466 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:46.832633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 727/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:46.837547 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef756f9-9c27-4d04-8dc2-ce6984118774 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:46.837970 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef756f9-9c27-4d04-8dc2-ce6984118774 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:46.838092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef756f9-9c27-4d04-8dc2-ce6984118774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:46.911555 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-aef756f9-9c27-4d04-8dc2-ce6984118774 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:46.912744 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-aef756f9-9c27-4d04-8dc2-ce6984118774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:46.952879 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:47.007688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 709/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:24:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:47.159781 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed7cc6-f535-45b1-b73d-5da3d21cd466 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:47.161309 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1aed7cc6-f535-45b1-b73d-5da3d21cd466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:47.304230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed7cc6-f535-45b1-b73d-5da3d21cd466 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:47.359426 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:47.420947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34f002-027a-459e-b882-553dfefcfea9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:47.421472 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34f002-027a-459e-b882-553dfefcfea9 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:47.421747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34f002-027a-459e-b882-553dfefcfea9 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:47.422016 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34f002-027a-459e-b882-553dfefcfea9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:47.423646 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-df34f002-027a-459e-b882-553dfefcfea9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:47.510217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df34f002-027a-459e-b882-553dfefcfea9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:47.512319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df34f002-027a-459e-b882-553dfefcfea9 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:47.530967 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 728/1437] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:24:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:24:48.345874 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfde70-d89a-47df-80f4-1a5602319aac None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:48.346473 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfde70-d89a-47df-80f4-1a5602319aac None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:48.346473 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfde70-d89a-47df-80f4-1a5602319aac None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:48.347806 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 710/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:48.352829 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225529c7-28e0-4982-9bc5-5c8bca218010 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:48.353216 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225529c7-28e0-4982-9bc5-5c8bca218010 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:48.353384 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225529c7-28e0-4982-9bc5-5c8bca218010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:48.429213 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-225529c7-28e0-4982-9bc5-5c8bca218010 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:48.430457 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-225529c7-28e0-4982-9bc5-5c8bca218010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:48.432469 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:48.527644 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 729/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:48.677896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfde70-d89a-47df-80f4-1a5602319aac None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:48.679479 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-74dfde70-d89a-47df-80f4-1a5602319aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:48.823200 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfde70-d89a-47df-80f4-1a5602319aac None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:48.849154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:48.935627 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb23001d-463b-454e-8d6c-40ed9970f860 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:48.936150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23001d-463b-454e-8d6c-40ed9970f860 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:48.936431 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23001d-463b-454e-8d6c-40ed9970f860 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:48.936705 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23001d-463b-454e-8d6c-40ed9970f860 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:48.938337 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bb23001d-463b-454e-8d6c-40ed9970f860 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:49.025874 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb23001d-463b-454e-8d6c-40ed9970f860 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:49.028018 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb23001d-463b-454e-8d6c-40ed9970f860 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:49.047521 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 711/1440] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:24:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:24:50.866001 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf26128-08fd-4fcf-a3f3-35d7ee589e79 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:50.866562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf26128-08fd-4fcf-a3f3-35d7ee589e79 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:50.866562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf26128-08fd-4fcf-a3f3-35d7ee589e79 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:50.867906 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 730/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:24:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:50.873679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77175a-0757-4ced-8afa-a8be92fff54c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:50.874077 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77175a-0757-4ced-8afa-a8be92fff54c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:50.874266 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77175a-0757-4ced-8afa-a8be92fff54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:50.914799 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:50.939962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0e77175a-0757-4ced-8afa-a8be92fff54c None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:50.941199 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0e77175a-0757-4ced-8afa-a8be92fff54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:51.035954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 712/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:51.332247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf26128-08fd-4fcf-a3f3-35d7ee589e79 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:51.333965 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1bf26128-08fd-4fcf-a3f3-35d7ee589e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:51.437262 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf26128-08fd-4fcf-a3f3-35d7ee589e79 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:51.468601 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:51.510934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a74692-8e8e-4210-a827-59eca34b2a36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:51.511591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a74692-8e8e-4210-a827-59eca34b2a36 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:51.511871 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a74692-8e8e-4210-a827-59eca34b2a36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:51.512143 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a74692-8e8e-4210-a827-59eca34b2a36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:51.513729 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-60a74692-8e8e-4210-a827-59eca34b2a36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:51.623157 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a74692-8e8e-4210-a827-59eca34b2a36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:51.624503 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a74692-8e8e-4210-a827-59eca34b2a36 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:51.642295 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 731/1443] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:24:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:24:53.715264 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cb0af-6ce1-4bdf-81d3-0b9f874d446f None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:53.715882 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cb0af-6ce1-4bdf-81d3-0b9f874d446f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:53.715882 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cb0af-6ce1-4bdf-81d3-0b9f874d446f None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:53.717174 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 713/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:53.722317 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588c896-276a-45bb-9ac4-d5295b4e60b8 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:53.722725 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588c896-276a-45bb-9ac4-d5295b4e60b8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:53.722862 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588c896-276a-45bb-9ac4-d5295b4e60b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:53.787503 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2588c896-276a-45bb-9ac4-d5295b4e60b8 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:53.788683 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2588c896-276a-45bb-9ac4-d5295b4e60b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:53.801873 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:53.886069 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 732/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:24:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:54.172185 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cb0af-6ce1-4bdf-81d3-0b9f874d446f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:54.173716 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a76cb0af-6ce1-4bdf-81d3-0b9f874d446f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:54.278335 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cb0af-6ce1-4bdf-81d3-0b9f874d446f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:54.290659 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:54.353090 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c1d92-0470-47ce-83e9-242e04f67ec0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:54.353796 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c1d92-0470-47ce-83e9-242e04f67ec0 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:54.354069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c1d92-0470-47ce-83e9-242e04f67ec0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:54.354371 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c1d92-0470-47ce-83e9-242e04f67ec0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:54.356604 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cb0c1d92-0470-47ce-83e9-242e04f67ec0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:54.466602 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c1d92-0470-47ce-83e9-242e04f67ec0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:54.468040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c1d92-0470-47ce-83e9-242e04f67ec0 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:54.486148 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 714/1446] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:24:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:24:56.727712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65723baf-b46e-4002-99a3-88572d810d9c None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:56.728298 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65723baf-b46e-4002-99a3-88572d810d9c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:56.728298 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65723baf-b46e-4002-99a3-88572d810d9c None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:56.729590 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 733/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:56.735022 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef7898-f6da-4849-a9a7-718ab6b01cf3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:56.735433 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef7898-f6da-4849-a9a7-718ab6b01cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:56.735570 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef7898-f6da-4849-a9a7-718ab6b01cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:56.784022 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:56.801767 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-adef7898-f6da-4849-a9a7-718ab6b01cf3 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:56.803001 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-adef7898-f6da-4849-a9a7-718ab6b01cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:56.898155 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 715/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:24:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:57.190242 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65723baf-b46e-4002-99a3-88572d810d9c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:57.191767 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-65723baf-b46e-4002-99a3-88572d810d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:57.297101 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65723baf-b46e-4002-99a3-88572d810d9c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:57.336302 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:57.371054 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5055bca-bc4b-4de6-bcb4-50eb3082fea1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:57.372258 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5055bca-bc4b-4de6-bcb4-50eb3082fea1 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:57.372534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5055bca-bc4b-4de6-bcb4-50eb3082fea1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:57.372803 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5055bca-bc4b-4de6-bcb4-50eb3082fea1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:57.374416 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c5055bca-bc4b-4de6-bcb4-50eb3082fea1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:57.480004 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5055bca-bc4b-4de6-bcb4-50eb3082fea1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:57.482251 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5055bca-bc4b-4de6-bcb4-50eb3082fea1 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:57.500881 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 734/1449] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:24:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:24:59.546490 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f52582-acdf-427c-9d5c-fa25ce1d6f22 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:59.547028 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f52582-acdf-427c-9d5c-fa25ce1d6f22 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:59.547028 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f52582-acdf-427c-9d5c-fa25ce1d6f22 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:59.548414 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 716/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:59.553656 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e6b9e-3988-4d88-93da-9a2989f0567a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:59.554068 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e6b9e-3988-4d88-93da-9a2989f0567a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:59.554234 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e6b9e-3988-4d88-93da-9a2989f0567a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:59.621311 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ef7e6b9e-3988-4d88-93da-9a2989f0567a None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:59.622462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ef7e6b9e-3988-4d88-93da-9a2989f0567a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:59.642944 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:59.720778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 735/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:24:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:00.014620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f52582-acdf-427c-9d5c-fa25ce1d6f22 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:00.016152 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b0f52582-acdf-427c-9d5c-fa25ce1d6f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:00.119367 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f52582-acdf-427c-9d5c-fa25ce1d6f22 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:00.139819 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:00.192822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cff7275-de67-4bee-b537-905931acec5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:00.193430 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff7275-de67-4bee-b537-905931acec5d service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:00.193771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff7275-de67-4bee-b537-905931acec5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:00.194044 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff7275-de67-4bee-b537-905931acec5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:00.195792 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8cff7275-de67-4bee-b537-905931acec5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:00.310272 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cff7275-de67-4bee-b537-905931acec5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:00.311620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cff7275-de67-4bee-b537-905931acec5d service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:00.329492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 717/1452] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:25:02.282146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0e8e0-f9fa-433e-8321-869972a3dae0 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:02.282733 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0e8e0-f9fa-433e-8321-869972a3dae0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:02.282733 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0e8e0-f9fa-433e-8321-869972a3dae0 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:02.284081 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 736/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:02.289147 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633a204-3440-4150-8bf4-23e9eaea11c4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:02.289565 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633a204-3440-4150-8bf4-23e9eaea11c4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:02.289693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633a204-3440-4150-8bf4-23e9eaea11c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:02.353128 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:02.355593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1633a204-3440-4150-8bf4-23e9eaea11c4 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:02.358003 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1633a204-3440-4150-8bf4-23e9eaea11c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:02.451792 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 718/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:25:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:02.733713 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f0e8e0-f9fa-433e-8321-869972a3dae0 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:02.735283 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f4f0e8e0-f9fa-433e-8321-869972a3dae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:02.837979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f0e8e0-f9fa-433e-8321-869972a3dae0 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:02.886315 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:02.911258 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00777102-f6ea-40fe-bf21-e2d102706dc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:02.911868 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00777102-f6ea-40fe-bf21-e2d102706dc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:02.912149 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00777102-f6ea-40fe-bf21-e2d102706dc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:02.912423 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00777102-f6ea-40fe-bf21-e2d102706dc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:02.914027 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-00777102-f6ea-40fe-bf21-e2d102706dc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:03.021396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00777102-f6ea-40fe-bf21-e2d102706dc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:03.022823 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00777102-f6ea-40fe-bf21-e2d102706dc1 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:03.040668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 737/1455] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:25:05.040484 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5b9b8-3d09-492f-936a-75383bf48660 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:05.041051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5b9b8-3d09-492f-936a-75383bf48660 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:05.041149 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5b9b8-3d09-492f-936a-75383bf48660 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:05.042475 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 719/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:05.047920 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a057b97-0596-49c8-a1c6-cdea10da7c24 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:05.048336 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a057b97-0596-49c8-a1c6-cdea10da7c24 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:05.048336 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a057b97-0596-49c8-a1c6-cdea10da7c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:05.122653 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2a057b97-0596-49c8-a1c6-cdea10da7c24 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:05.123868 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2a057b97-0596-49c8-a1c6-cdea10da7c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:05.153904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:05.221156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 738/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:05.509368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5b9b8-3d09-492f-936a-75383bf48660 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:05.510983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-49b5b9b8-3d09-492f-936a-75383bf48660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:05.614738 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5b9b8-3d09-492f-936a-75383bf48660 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:05.644702 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:05.691675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10270a71-2c26-4d07-8918-63f64f43ce3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:05.692284 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10270a71-2c26-4d07-8918-63f64f43ce3a service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:05.692570 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10270a71-2c26-4d07-8918-63f64f43ce3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:05.692971 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10270a71-2c26-4d07-8918-63f64f43ce3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:05.694582 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-10270a71-2c26-4d07-8918-63f64f43ce3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:05.806687 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10270a71-2c26-4d07-8918-63f64f43ce3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:05.808128 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10270a71-2c26-4d07-8918-63f64f43ce3a service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:05.826295 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 720/1458] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:25:07.753215 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d7639-6f70-46bd-841d-bba747b77420 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:07.753783 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d7639-6f70-46bd-841d-bba747b77420 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:07.753783 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d7639-6f70-46bd-841d-bba747b77420 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:07.762802 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 739/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:25:07 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:07.770435 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037859cd-a10e-434e-a1fb-45226c577887 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:07.770966 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037859cd-a10e-434e-a1fb-45226c577887 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:07.771064 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037859cd-a10e-434e-a1fb-45226c577887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:07.838243 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-037859cd-a10e-434e-a1fb-45226c577887 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:07.839452 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-037859cd-a10e-434e-a1fb-45226c577887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:07.847654 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:07.936642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 721/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:25:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:25:08.235151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d7639-6f70-46bd-841d-bba747b77420 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:08.236727 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e59d7639-6f70-46bd-841d-bba747b77420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:08.339881 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d7639-6f70-46bd-841d-bba747b77420 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:08.394060 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:08.412214 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb4db0-20dd-4a60-afb3-87976f9fb86e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:08.412810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb4db0-20dd-4a60-afb3-87976f9fb86e service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:08.413083 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb4db0-20dd-4a60-afb3-87976f9fb86e service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:08.413350 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb4db0-20dd-4a60-afb3-87976f9fb86e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:08.414955 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-fadb4db0-20dd-4a60-afb3-87976f9fb86e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:08.539046 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadb4db0-20dd-4a60-afb3-87976f9fb86e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:08.540512 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadb4db0-20dd-4a60-afb3-87976f9fb86e service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:08.558479 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 740/1461] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:25:10.559304 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18400cea-c40e-4cc9-9adc-77997560e69e None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:10.559872 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18400cea-c40e-4cc9-9adc-77997560e69e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:10.560123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18400cea-c40e-4cc9-9adc-77997560e69e None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:10.561525 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 722/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:10.566691 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da47a22-1530-4928-93e5-a618e7d26cd8 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:10.567110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da47a22-1530-4928-93e5-a618e7d26cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:10.567242 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da47a22-1530-4928-93e5-a618e7d26cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:10.633162 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-0da47a22-1530-4928-93e5-a618e7d26cd8 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:10.634380 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0da47a22-1530-4928-93e5-a618e7d26cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:10.672246 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:10.728511 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 741/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:11.012072 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18400cea-c40e-4cc9-9adc-77997560e69e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:11.014133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-18400cea-c40e-4cc9-9adc-77997560e69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:11.116999 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18400cea-c40e-4cc9-9adc-77997560e69e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:11.154700 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:11.190651 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c614853-a657-453c-bec8-26436bde93c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:11.191258 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c614853-a657-453c-bec8-26436bde93c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:11.191543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c614853-a657-453c-bec8-26436bde93c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:11.191820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c614853-a657-453c-bec8-26436bde93c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:11.193495 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0c614853-a657-453c-bec8-26436bde93c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:11.302611 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c614853-a657-453c-bec8-26436bde93c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:11.303964 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c614853-a657-453c-bec8-26436bde93c2 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:11.322159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 723/1464] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:25:13.258612 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379df7c-b775-4380-89ae-6f55bb5c289e None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:13.259146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379df7c-b775-4380-89ae-6f55bb5c289e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:13.259146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379df7c-b775-4380-89ae-6f55bb5c289e None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:13.260499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 742/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:13.266319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96f06f-5b5d-442a-87df-abcb1d4e358c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:13.266713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96f06f-5b5d-442a-87df-abcb1d4e358c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:13.266884 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96f06f-5b5d-442a-87df-abcb1d4e358c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:13.338886 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6c96f06f-5b5d-442a-87df-abcb1d4e358c None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:13.340186 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6c96f06f-5b5d-442a-87df-abcb1d4e358c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:13.358892 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:13.442740 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 724/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:13.736245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b379df7c-b775-4380-89ae-6f55bb5c289e None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:13.737798 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b379df7c-b775-4380-89ae-6f55bb5c289e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:13.980961 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b379df7c-b775-4380-89ae-6f55bb5c289e None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:14.049210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:14.056519 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b136a7-085f-48de-b6f5-df8fe5a4a3d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:14.057122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b136a7-085f-48de-b6f5-df8fe5a4a3d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:14.057399 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b136a7-085f-48de-b6f5-df8fe5a4a3d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:14.057671 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b136a7-085f-48de-b6f5-df8fe5a4a3d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:14.059377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a6b136a7-085f-48de-b6f5-df8fe5a4a3d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:14.167280 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b136a7-085f-48de-b6f5-df8fe5a4a3d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:14.169426 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b136a7-085f-48de-b6f5-df8fe5a4a3d0 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:14.187090 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 743/1467] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:25:16.100727 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4c2aa-4df4-4b43-9411-08a87f2d23cb None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:16.101312 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4c2aa-4df4-4b43-9411-08a87f2d23cb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:16.101312 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4c2aa-4df4-4b43-9411-08a87f2d23cb None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:16.102677 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 725/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:16.108454 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb424ee3-f464-4a92-af7d-d3eb7b9c1fe4 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:16.108864 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb424ee3-f464-4a92-af7d-d3eb7b9c1fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:16.109007 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb424ee3-f464-4a92-af7d-d3eb7b9c1fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:16.174719 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-bb424ee3-f464-4a92-af7d-d3eb7b9c1fe4 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:16.175924 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bb424ee3-f464-4a92-af7d-d3eb7b9c1fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:16.222571 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:16.278111 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 744/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:16.565789 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4c2aa-4df4-4b43-9411-08a87f2d23cb None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:16.567359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-55d4c2aa-4df4-4b43-9411-08a87f2d23cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:16.669774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4c2aa-4df4-4b43-9411-08a87f2d23cb None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:16.715965 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:16.742365 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71411909-e491-4550-8e28-1c253501c6a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:16.742966 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71411909-e491-4550-8e28-1c253501c6a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:16.743249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71411909-e491-4550-8e28-1c253501c6a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:16.743522 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71411909-e491-4550-8e28-1c253501c6a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:16.745125 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-71411909-e491-4550-8e28-1c253501c6a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:16.853145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71411909-e491-4550-8e28-1c253501c6a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:16.854533 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71411909-e491-4550-8e28-1c253501c6a8 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:16.872080 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 726/1470] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:25:26.276312 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=22 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:26.869817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=66 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:36.335649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83654fa-8490-43ad-a5ab-4ce968ccc251 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:36.336173 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83654fa-8490-43ad-a5ab-4ce968ccc251 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:36.336173 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83654fa-8490-43ad-a5ab-4ce968ccc251 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:36.337493 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 745/1471] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:36.342550 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3a8ec-6ef5-4348-ad44-e8b25a21dc76 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:36.342981 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3a8ec-6ef5-4348-ad44-e8b25a21dc76 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:36.343081 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3a8ec-6ef5-4348-ad44-e8b25a21dc76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:36.406699 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0ea3a8ec-6ef5-4348-ad44-e8b25a21dc76 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:36.408144 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0ea3a8ec-6ef5-4348-ad44-e8b25a21dc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:36.501727 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 727/1472] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:36.798399 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421b55a-d7ad-4e72-89b7-fb250f7d7c35 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:36.798683 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421b55a-d7ad-4e72-89b7-fb250f7d7c35 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:36.798957 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421b55a-d7ad-4e72-89b7-fb250f7d7c35 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:36.800290 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 746/1473] 10.222.0.58 () {54 vars in 967 bytes} [Thu Sep 1 16:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:36.805326 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771921f5-87c4-4d8e-8e58-cdb1914558e1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:36.805609 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771921f5-87c4-4d8e-8e58-cdb1914558e1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:36.805882 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771921f5-87c4-4d8e-8e58-cdb1914558e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:36.880105 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-771921f5-87c4-4d8e-8e58-cdb1914558e1 None None] MFA Rules not processed for user `6a2c872129414105a112661cc2b477e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:36.881295 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-771921f5-87c4-4d8e-8e58-cdb1914558e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:36.900970 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:36.955321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 728/1474] 10.222.0.58 () {58 vars in 1064 bytes} [Thu Sep 1 16:25:36 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:36.962637 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421b55a-d7ad-4e72-89b7-fb250f7d7c35 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:36.964183 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f421b55a-d7ad-4e72-89b7-fb250f7d7c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:37.067165 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421b55a-d7ad-4e72-89b7-fb250f7d7c35 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:37.140020 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e04bb-2d3f-45a5-8e80-3438cfdf9994 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2c872129414105a112661cc2b477e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:37.140617 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e04bb-2d3f-45a5-8e80-3438cfdf9994 service glance] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:37.140895 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e04bb-2d3f-45a5-8e80-3438cfdf9994 service glance] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:37.141166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e04bb-2d3f-45a5-8e80-3438cfdf9994 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:37.142783 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-db1e04bb-2d3f-45a5-8e80-3438cfdf9994 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:37.196059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:37.250515 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e04bb-2d3f-45a5-8e80-3438cfdf9994 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:37.251941 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e04bb-2d3f-45a5-8e80-3438cfdf9994 service glance] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:37.270595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 747/1475] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:25:39.304432 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2451b-4fc0-49e3-b03a-85a3ee3baf73 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:39.304981 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2451b-4fc0-49e3-b03a-85a3ee3baf73 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:39.305206 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2451b-4fc0-49e3-b03a-85a3ee3baf73 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:39.307364 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 729/1476] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:25:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:39.313955 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03bbb45-5fc6-415c-8093-9362f5d5b425 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:39.314400 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03bbb45-5fc6-415c-8093-9362f5d5b425 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:39.314557 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03bbb45-5fc6-415c-8093-9362f5d5b425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:39.379896 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e03bbb45-5fc6-415c-8093-9362f5d5b425 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:39.381109 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e03bbb45-5fc6-415c-8093-9362f5d5b425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:39.474740 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 748/1477] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:39.752278 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2451b-4fc0-49e3-b03a-85a3ee3baf73 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:39.753804 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-62f2451b-4fc0-49e3-b03a-85a3ee3baf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:39.877251 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2451b-4fc0-49e3-b03a-85a3ee3baf73 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:39.926992 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:39.985825 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5719639-2551-4084-a693-f3a00559bfb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:39.986929 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5719639-2551-4084-a693-f3a00559bfb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:39.987321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5719639-2551-4084-a693-f3a00559bfb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:39.987658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5719639-2551-4084-a693-f3a00559bfb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:39.988868 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5719639-2551-4084-a693-f3a00559bfb2 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:39.991358 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-d5719639-2551-4084-a693-f3a00559bfb2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:25:39.991358 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Sep 01 16:25:39.991358 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: Sep 01 16:25:39.992890 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5719639-2551-4084-a693-f3a00559bfb2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:40.016786 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 730/1478] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Sep 1 16:25:39 2022] GET /identity/v3/projects => generated 2553 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:25:41.930386 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c9cb7-3f45-4767-93b7-c7c4ab9f46ed None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:41.930958 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c9cb7-3f45-4767-93b7-c7c4ab9f46ed None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:41.930958 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c9cb7-3f45-4767-93b7-c7c4ab9f46ed None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:41.932251 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 749/1479] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:41.937396 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400505a4-c058-4d8a-9550-ec3c7d9fc487 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:41.937805 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400505a4-c058-4d8a-9550-ec3c7d9fc487 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:41.937956 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400505a4-c058-4d8a-9550-ec3c7d9fc487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:42.004144 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-400505a4-c058-4d8a-9550-ec3c7d9fc487 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:42.005532 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-400505a4-c058-4d8a-9550-ec3c7d9fc487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:42.104941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 731/1480] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:42.394027 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c9cb7-3f45-4767-93b7-c7c4ab9f46ed None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:42.395673 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-525c9cb7-3f45-4767-93b7-c7c4ab9f46ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:42.406580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:42.539467 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c9cb7-3f45-4767-93b7-c7c4ab9f46ed None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:42.617632 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:42.649539 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6f508-fd1d-46dd-9161-228fc5a06d79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:42.650062 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6f508-fd1d-46dd-9161-228fc5a06d79 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:42.650363 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6f508-fd1d-46dd-9161-228fc5a06d79 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:42.650634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6f508-fd1d-46dd-9161-228fc5a06d79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:42.652195 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0ba6f508-fd1d-46dd-9161-228fc5a06d79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:42.739987 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba6f508-fd1d-46dd-9161-228fc5a06d79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:42.741589 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba6f508-fd1d-46dd-9161-228fc5a06d79 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:42.758976 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 750/1481] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:25:44.850098 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5c7a5-0fcc-4f9c-a68f-d43bacf633cd None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:44.850703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5c7a5-0fcc-4f9c-a68f-d43bacf633cd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:44.850703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5c7a5-0fcc-4f9c-a68f-d43bacf633cd None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:44.852056 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 732/1482] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:44.857297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a58853-0a3e-4afa-8787-af04c3f03727 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:44.857706 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a58853-0a3e-4afa-8787-af04c3f03727 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:44.857870 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a58853-0a3e-4afa-8787-af04c3f03727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:44.923340 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e7a58853-0a3e-4afa-8787-af04c3f03727 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:44.924539 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e7a58853-0a3e-4afa-8787-af04c3f03727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:44.970462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:45.020652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 751/1483] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:45.314135 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5c7a5-0fcc-4f9c-a68f-d43bacf633cd None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:45.315706 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-28e5c7a5-0fcc-4f9c-a68f-d43bacf633cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:45.337978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:45.418108 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5c7a5-0fcc-4f9c-a68f-d43bacf633cd None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:45.490986 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39ef9e-aade-4e1f-a3a1-810207e15130 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:45.491582 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39ef9e-aade-4e1f-a3a1-810207e15130 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:45.491871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39ef9e-aade-4e1f-a3a1-810207e15130 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:45.492143 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39ef9e-aade-4e1f-a3a1-810207e15130 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:45.493733 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bc39ef9e-aade-4e1f-a3a1-810207e15130 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:45.567247 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:45.601372 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39ef9e-aade-4e1f-a3a1-810207e15130 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:45.602861 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39ef9e-aade-4e1f-a3a1-810207e15130 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:45.623573 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 733/1484] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:25:47.557126 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e05eb0-cbda-47d4-bfc3-a62624be6958 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:47.557749 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e05eb0-cbda-47d4-bfc3-a62624be6958 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:47.557749 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e05eb0-cbda-47d4-bfc3-a62624be6958 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:47.559029 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 752/1485] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:47.564294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4c241-f2ba-4fa0-9353-6b4a39a37198 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:47.564751 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4c241-f2ba-4fa0-9353-6b4a39a37198 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:47.564860 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4c241-f2ba-4fa0-9353-6b4a39a37198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:47.629886 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6ad4c241-f2ba-4fa0-9353-6b4a39a37198 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:47.631048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6ad4c241-f2ba-4fa0-9353-6b4a39a37198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:47.726326 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 734/1486] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:48.014585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e05eb0-cbda-47d4-bfc3-a62624be6958 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:48.016146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-88e05eb0-cbda-47d4-bfc3-a62624be6958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:48.118834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e05eb0-cbda-47d4-bfc3-a62624be6958 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:48.188690 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:48.190424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593640aa-dd99-4fa5-99c3-2eeea22ef0cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:48.191034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593640aa-dd99-4fa5-99c3-2eeea22ef0cb service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:48.191554 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593640aa-dd99-4fa5-99c3-2eeea22ef0cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:48.191868 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593640aa-dd99-4fa5-99c3-2eeea22ef0cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:48.193812 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-593640aa-dd99-4fa5-99c3-2eeea22ef0cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:48.302497 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593640aa-dd99-4fa5-99c3-2eeea22ef0cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:48.303841 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593640aa-dd99-4fa5-99c3-2eeea22ef0cb service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:48.321443 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 753/1487] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:25:50.312114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1d2fc-ce7e-4f16-bca2-da49e1716997 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:50.312722 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1d2fc-ce7e-4f16-bca2-da49e1716997 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:50.312722 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1d2fc-ce7e-4f16-bca2-da49e1716997 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:50.314290 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 735/1488] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:25:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:50.319462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9894844-16e3-4608-893d-8a2d3c6fde93 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:50.319904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9894844-16e3-4608-893d-8a2d3c6fde93 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:50.320011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9894844-16e3-4608-893d-8a2d3c6fde93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:50.386486 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b9894844-16e3-4608-893d-8a2d3c6fde93 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:50.387682 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b9894844-16e3-4608-893d-8a2d3c6fde93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:50.441123 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:50.485913 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 754/1489] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:50.773839 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e1d2fc-ce7e-4f16-bca2-da49e1716997 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:50.775433 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-61e1d2fc-ce7e-4f16-bca2-da49e1716997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:50.811115 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:50.879849 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e1d2fc-ce7e-4f16-bca2-da49e1716997 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:50.952756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeaf95a-e1b6-4d24-b2e9-d7a49bc3e36b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:50.953361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeaf95a-e1b6-4d24-b2e9-d7a49bc3e36b service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:50.953645 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeaf95a-e1b6-4d24-b2e9-d7a49bc3e36b service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:50.953936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeaf95a-e1b6-4d24-b2e9-d7a49bc3e36b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:50.955712 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9aeaf95a-e1b6-4d24-b2e9-d7a49bc3e36b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:51.034944 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:51.064158 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeaf95a-e1b6-4d24-b2e9-d7a49bc3e36b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:51.065596 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeaf95a-e1b6-4d24-b2e9-d7a49bc3e36b service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:51.083955 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 736/1490] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:25:53.203623 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35d88e-c092-42cb-b42d-f14c236ffe7f None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:53.204202 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35d88e-c092-42cb-b42d-f14c236ffe7f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:53.204202 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35d88e-c092-42cb-b42d-f14c236ffe7f None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:53.205530 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 755/1491] 10.222.0.58 () {54 vars in 927 bytes} [Thu Sep 1 16:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:53.210914 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e830a00-82b2-4c96-be7e-ae7c5fae9c03 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:53.211329 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e830a00-82b2-4c96-be7e-ae7c5fae9c03 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:53.211475 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e830a00-82b2-4c96-be7e-ae7c5fae9c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:53.277539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1e830a00-82b2-4c96-be7e-ae7c5fae9c03 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:53.279262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1e830a00-82b2-4c96-be7e-ae7c5fae9c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:53.387652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 737/1492] 10.222.0.58 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:53.684154 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35d88e-c092-42cb-b42d-f14c236ffe7f None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:53.685722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bb35d88e-c092-42cb-b42d-f14c236ffe7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:53.789814 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35d88e-c092-42cb-b42d-f14c236ffe7f None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:53.864904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec7c15-5291-4712-b3b0-3db703b43312 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:53.865505 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec7c15-5291-4712-b3b0-3db703b43312 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:53.865785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec7c15-5291-4712-b3b0-3db703b43312 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:53.866060 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec7c15-5291-4712-b3b0-3db703b43312 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:53.867699 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-faec7c15-5291-4712-b3b0-3db703b43312 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:53.877249 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:53.976435 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faec7c15-5291-4712-b3b0-3db703b43312 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:53.977760 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faec7c15-5291-4712-b3b0-3db703b43312 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:53.995114 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 756/1493] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:03.378330 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=82 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:03.992907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=50 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:06.603580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6c03f-6182-4eb2-9b5c-c58fe90e615c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:06.604203 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6c03f-6182-4eb2-9b5c-c58fe90e615c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:06.604203 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6c03f-6182-4eb2-9b5c-c58fe90e615c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:06.679101 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3ef6c03f-6182-4eb2-9b5c-c58fe90e615c None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:06.680307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3ef6c03f-6182-4eb2-9b5c-c58fe90e615c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:06.775655 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 738/1494] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:27:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:06.783861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f75b56-ee6c-4264-97d6-536807f6b049 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:06.784346 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f75b56-ee6c-4264-97d6-536807f6b049 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:06.784456 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f75b56-ee6c-4264-97d6-536807f6b049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:06.858529 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b2f75b56-ee6c-4264-97d6-536807f6b049 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:06.859663 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b2f75b56-ee6c-4264-97d6-536807f6b049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:06.954881 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 757/1495] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:27:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:06.959462 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef6c03f-6182-4eb2-9b5c-c58fe90e615c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:06.961071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3ef6c03f-6182-4eb2-9b5c-c58fe90e615c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:06.972318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef6c03f-6182-4eb2-9b5c-c58fe90e615c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:07.032353 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:07.069631 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb43975-5422-4d54-894b-0749aecbb5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:07.070272 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb43975-5422-4d54-894b-0749aecbb5da admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:07.070557 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb43975-5422-4d54-894b-0749aecbb5da admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:07.070832 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb43975-5422-4d54-894b-0749aecbb5da admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:07.071483 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb43975-5422-4d54-894b-0749aecbb5da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:27:07.071792 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb43975-5422-4d54-894b-0749aecbb5da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:07.072808 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-1fb43975-5422-4d54-894b-0749aecbb5da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:27:07.072808 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Sep 01 16:27:07.072808 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: Sep 01 16:27:07.073715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb43975-5422-4d54-894b-0749aecbb5da admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:07.085287 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 739/1496] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:27:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:27:07.094163 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f75b56-ee6c-4264-97d6-536807f6b049 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:07.103163 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f75b56-ee6c-4264-97d6-536807f6b049 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:07.163228 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:07.196362 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721af9cd-d6cd-4a06-b5a7-8f375606c19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:07.196959 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721af9cd-d6cd-4a06-b5a7-8f375606c19b admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:07.197237 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721af9cd-d6cd-4a06-b5a7-8f375606c19b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:07.197535 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721af9cd-d6cd-4a06-b5a7-8f375606c19b admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:07.198169 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721af9cd-d6cd-4a06-b5a7-8f375606c19b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:07.199939 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: WARNING py.warnings [None req-721af9cd-d6cd-4a06-b5a7-8f375606c19b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:27:07.199939 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: warnings.warn(msg) Sep 01 16:27:07.199939 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: Sep 01 16:27:07.201165 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721af9cd-d6cd-4a06-b5a7-8f375606c19b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:07.256037 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 758/1497] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:27:07 2022] POST /identity/v3/projects => generated 379 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:27:07.260262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb43975-5422-4d54-894b-0749aecbb5da admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:07.261827 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1fb43975-5422-4d54-894b-0749aecbb5da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:07.296011 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb43975-5422-4d54-894b-0749aecbb5da admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:07.387458 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb850fb-f988-45c7-98d9-b9ff3d47ad70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:07.388057 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb850fb-f988-45c7-98d9-b9ff3d47ad70 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:07.388339 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb850fb-f988-45c7-98d9-b9ff3d47ad70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:07.388612 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb850fb-f988-45c7-98d9-b9ff3d47ad70 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:07.393380 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb850fb-f988-45c7-98d9-b9ff3d47ad70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:07.393380 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-6fb850fb-f988-45c7-98d9-b9ff3d47ad70 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:27:07.393380 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Sep 01 16:27:07.393380 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: Sep 01 16:27:07.393380 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb850fb-f988-45c7-98d9-b9ff3d47ad70 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:07.440073 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 740/1498] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:27:07 2022] POST /identity/v3/users => generated 290 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:27:07.444187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721af9cd-d6cd-4a06-b5a7-8f375606c19b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:07.445753 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-721af9cd-d6cd-4a06-b5a7-8f375606c19b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:07.507481 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721af9cd-d6cd-4a06-b5a7-8f375606c19b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:07.593237 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:07.605308 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bad0c9-4998-43d1-a555-e1a1e66074ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:07.605904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bad0c9-4998-43d1-a555-e1a1e66074ce admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:07.606179 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bad0c9-4998-43d1-a555-e1a1e66074ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:07.606472 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bad0c9-4998-43d1-a555-e1a1e66074ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:07.607112 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bad0c9-4998-43d1-a555-e1a1e66074ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:07.608706 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: WARNING py.warnings [None req-34bad0c9-4998-43d1-a555-e1a1e66074ce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:27:07.608706 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: warnings.warn(msg) Sep 01 16:27:07.608706 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: Sep 01 16:27:07.609574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bad0c9-4998-43d1-a555-e1a1e66074ce admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:07.621531 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 759/1499] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:27:07.625617 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb850fb-f988-45c7-98d9-b9ff3d47ad70 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:07.627216 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6fb850fb-f988-45c7-98d9-b9ff3d47ad70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:07.638690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb850fb-f988-45c7-98d9-b9ff3d47ad70 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:07.666669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:07.744704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65121824-f229-4e47-9765-70dff29f12b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:07.745225 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65121824-f229-4e47-9765-70dff29f12b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:07.745508 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65121824-f229-4e47-9765-70dff29f12b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:07.745785 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65121824-f229-4e47-9765-70dff29f12b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:07.746483 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65121824-f229-4e47-9765-70dff29f12b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:07.747482 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-65121824-f229-4e47-9765-70dff29f12b7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:27:07.747482 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Sep 01 16:27:07.747482 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: Sep 01 16:27:07.748375 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65121824-f229-4e47-9765-70dff29f12b7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:07.760579 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 741/1500] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:27:07.765322 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bad0c9-4998-43d1-a555-e1a1e66074ce admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:07.766852 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-34bad0c9-4998-43d1-a555-e1a1e66074ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:07.800377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bad0c9-4998-43d1-a555-e1a1e66074ce admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:07.893122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e552fcd-64f6-499c-8a3f-5b78466f4e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:07.893767 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e552fcd-64f6-499c-8a3f-5b78466f4e41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:07.894045 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e552fcd-64f6-499c-8a3f-5b78466f4e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:07.894348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e552fcd-64f6-499c-8a3f-5b78466f4e41 admin admin] PATH_INFO: `/v3/projects/200e15e3c5fb45e3a69055320b05a39c/users/878bc162e0304b1a86d0993c78866364/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:07.916550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e552fcd-64f6-499c-8a3f-5b78466f4e41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=200e15e3c5fb45e3a69055320b05a39c, user_id=878bc162e0304b1a86d0993c78866364, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:07.917574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: WARNING py.warnings [None req-7e552fcd-64f6-499c-8a3f-5b78466f4e41 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:27:07.917574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: warnings.warn(msg) Sep 01 16:27:07.917574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: Sep 01 16:27:07.919562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e552fcd-64f6-499c-8a3f-5b78466f4e41 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:07.943618 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 760/1501] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:27:07 2022] PUT /identity/v3/projects/200e15e3c5fb45e3a69055320b05a39c/users/878bc162e0304b1a86d0993c78866364/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:27:07.956115 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af191e0-7ccd-432f-bdcf-b956761b942a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:07.956405 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af191e0-7ccd-432f-bdcf-b956761b942a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:07.956681 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af191e0-7ccd-432f-bdcf-b956761b942a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:08.015469 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3af191e0-7ccd-432f-bdcf-b956761b942a None None] MFA Rules not processed for user `878bc162e0304b1a86d0993c78866364`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:08.016587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3af191e0-7ccd-432f-bdcf-b956761b942a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:08.095525 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 742/1502] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:27:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:08.108503 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a67504-c9ee-4169-8790-2cebe290c0f4 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:08.108786 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a67504-c9ee-4169-8790-2cebe290c0f4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:08.109147 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a67504-c9ee-4169-8790-2cebe290c0f4 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:08.110534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 761/1503] 10.222.0.58 () {54 vars in 964 bytes} [Thu Sep 1 16:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:08.116882 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cb11f-b529-4f44-a09c-cee6eb9e704e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:08.117163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cb11f-b529-4f44-a09c-cee6eb9e704e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:08.117440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cb11f-b529-4f44-a09c-cee6eb9e704e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:08.148888 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:08.192457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-dd8cb11f-b529-4f44-a09c-cee6eb9e704e None None] MFA Rules not processed for user `84d4bd0a456341f78b5e476c97a9165c`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:08.193646 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dd8cb11f-b529-4f44-a09c-cee6eb9e704e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:08.267385 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 743/1504] 10.222.0.58 () {58 vars in 1061 bytes} [Thu Sep 1 16:27:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:08.277194 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a67504-c9ee-4169-8790-2cebe290c0f4 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:08.278771 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-24a67504-c9ee-4169-8790-2cebe290c0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:08.328865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:08.387534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a67504-c9ee-4169-8790-2cebe290c0f4 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:08.460221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24767c5a-eec3-41db-90b4-8dea096cb1da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:08.460813 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24767c5a-eec3-41db-90b4-8dea096cb1da service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:08.461095 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24767c5a-eec3-41db-90b4-8dea096cb1da service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:08.461365 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24767c5a-eec3-41db-90b4-8dea096cb1da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:08.462990 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-24767c5a-eec3-41db-90b4-8dea096cb1da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:08.562932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24767c5a-eec3-41db-90b4-8dea096cb1da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:08.564587 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24767c5a-eec3-41db-90b4-8dea096cb1da service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:08.568075 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:08.583253 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 762/1505] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:08 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:08.627894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf392a32-35b4-494d-a1e1-f32890440855 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:08.628297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf392a32-35b4-494d-a1e1-f32890440855 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:08.628458 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf392a32-35b4-494d-a1e1-f32890440855 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:08.629717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 744/1506] 10.222.0.58 () {52 vars in 830 bytes} [Thu Sep 1 16:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:08.633181 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24767c5a-eec3-41db-90b4-8dea096cb1da service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:08.634415 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-24767c5a-eec3-41db-90b4-8dea096cb1da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:08.733562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24767c5a-eec3-41db-90b4-8dea096cb1da service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:08.823956 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb19e7d-812e-4673-906b-649cc1b10eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:08.824594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb19e7d-812e-4673-906b-649cc1b10eef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:08.824757 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb19e7d-812e-4673-906b-649cc1b10eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:08.825021 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb19e7d-812e-4673-906b-649cc1b10eef admin admin] PATH_INFO: `/v3/users/878bc162e0304b1a86d0993c78866364` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:08.837293 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb19e7d-812e-4673-906b-649cc1b10eef admin admin] RBAC: Authorizing `identity:delete_user(user_id=878bc162e0304b1a86d0993c78866364)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:08.838339 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: WARNING py.warnings [None req-2bb19e7d-812e-4673-906b-649cc1b10eef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:27:08.838339 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: warnings.warn(msg) Sep 01 16:27:08.838339 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: Sep 01 16:27:08.839548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb19e7d-812e-4673-906b-649cc1b10eef admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:08.848849 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:08.903819 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2bb19e7d-812e-4673-906b-649cc1b10eef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '878bc162e0304b1a86d0993c78866364'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:27:08.917685 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2bb19e7d-812e-4673-906b-649cc1b10eef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '878bc162e0304b1a86d0993c78866364'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:27:08.918130 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2bb19e7d-812e-4673-906b-649cc1b10eef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '878bc162e0304b1a86d0993c78866364'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:27:08.918689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2bb19e7d-812e-4673-906b-649cc1b10eef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '878bc162e0304b1a86d0993c78866364'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:27:08.941400 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2bb19e7d-812e-4673-906b-649cc1b10eef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '878bc162e0304b1a86d0993c78866364'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:27:08.956266 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 763/1507] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:27:08 2022] DELETE /identity/v3/users/878bc162e0304b1a86d0993c78866364 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:27:08.960716 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b9f3f-cf3d-4f25-ac99-72d9a06a0e80 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:08.961000 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b9f3f-cf3d-4f25-ac99-72d9a06a0e80 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:08.961276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b9f3f-cf3d-4f25-ac99-72d9a06a0e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:09.017462 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-416b9f3f-cf3d-4f25-ac99-72d9a06a0e80 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:09.018686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-416b9f3f-cf3d-4f25-ac99-72d9a06a0e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:09.093862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:09.106623 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 745/1508] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:09.116147 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb19e7d-812e-4673-906b-649cc1b10eef admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:09.117362 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2bb19e7d-812e-4673-906b-649cc1b10eef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:09.215915 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb19e7d-812e-4673-906b-649cc1b10eef admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:09.271683 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:09.292369 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47481e20-74d3-45b5-8d5d-1af707700220 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:09.292979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47481e20-74d3-45b5-8d5d-1af707700220 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:09.293258 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47481e20-74d3-45b5-8d5d-1af707700220 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:09.293530 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47481e20-74d3-45b5-8d5d-1af707700220 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:09.295172 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-47481e20-74d3-45b5-8d5d-1af707700220 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:09.406775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47481e20-74d3-45b5-8d5d-1af707700220 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:09.408118 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47481e20-74d3-45b5-8d5d-1af707700220 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:09.427305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 764/1509] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:09.785193 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b9f3f-cf3d-4f25-ac99-72d9a06a0e80 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:09.787107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-416b9f3f-cf3d-4f25-ac99-72d9a06a0e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:09.799302 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b9f3f-cf3d-4f25-ac99-72d9a06a0e80 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:09.896601 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fdff54-ead8-47d4-acfe-c9462635ae38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:09.897231 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdff54-ead8-47d4-acfe-c9462635ae38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:09.897513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdff54-ead8-47d4-acfe-c9462635ae38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:09.897786 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdff54-ead8-47d4-acfe-c9462635ae38 admin admin] PATH_INFO: `/v3/projects/200e15e3c5fb45e3a69055320b05a39c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:09.907991 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fdff54-ead8-47d4-acfe-c9462635ae38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=200e15e3c5fb45e3a69055320b05a39c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:09.909012 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-d8fdff54-ead8-47d4-acfe-c9462635ae38 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:27:09.909012 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Sep 01 16:27:09.909012 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: Sep 01 16:27:09.910300 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fdff54-ead8-47d4-acfe-c9462635ae38 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:09.970034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d8fdff54-ead8-47d4-acfe-c9462635ae38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '200e15e3c5fb45e3a69055320b05a39c', 'request_id': 'req-d8fdff54-ead8-47d4-acfe-c9462635ae38'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:27:09.983076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d8fdff54-ead8-47d4-acfe-c9462635ae38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '200e15e3c5fb45e3a69055320b05a39c', 'request_id': 'req-d8fdff54-ead8-47d4-acfe-c9462635ae38'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:27:09.993090 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d8fdff54-ead8-47d4-acfe-c9462635ae38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '200e15e3c5fb45e3a69055320b05a39c', 'request_id': 'req-d8fdff54-ead8-47d4-acfe-c9462635ae38'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:27:10.017274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d8fdff54-ead8-47d4-acfe-c9462635ae38 admin admin] The token cache is being invalidate because project 200e15e3c5fb45e3a69055320b05a39c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:27:10.017563 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d8fdff54-ead8-47d4-acfe-c9462635ae38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:27:10.017994 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d8fdff54-ead8-47d4-acfe-c9462635ae38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:27:10.019451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 746/1510] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:27:09 2022] DELETE /identity/v3/projects/200e15e3c5fb45e3a69055320b05a39c => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:27:19.425023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=62 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:20.014971 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:42.441641 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e7d3c-72a9-4233-b85b-e9c5ce3b48cf None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:42.442372 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e7d3c-72a9-4233-b85b-e9c5ce3b48cf None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:42.442372 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e7d3c-72a9-4233-b85b-e9c5ce3b48cf None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:42.443559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 765/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:42.448488 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f74613-cd5d-4eb6-b0be-6cdec7979e5f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:42.448927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f74613-cd5d-4eb6-b0be-6cdec7979e5f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:42.449034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f74613-cd5d-4eb6-b0be-6cdec7979e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:42.530990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-69f74613-cd5d-4eb6-b0be-6cdec7979e5f None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:42.532189 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-69f74613-cd5d-4eb6-b0be-6cdec7979e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:42.634375 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 747/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:28:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:42.788660 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e7d3c-72a9-4233-b85b-e9c5ce3b48cf None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:42.790827 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-091e7d3c-72a9-4233-b85b-e9c5ce3b48cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:42.915362 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e7d3c-72a9-4233-b85b-e9c5ce3b48cf None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:42.920881 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9f33f-df2f-49cd-8e13-cebddab0d941 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:42.921486 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9f33f-df2f-49cd-8e13-cebddab0d941 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:42.921768 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9f33f-df2f-49cd-8e13-cebddab0d941 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:42.922042 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9f33f-df2f-49cd-8e13-cebddab0d941 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:42.930493 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9f33f-df2f-49cd-8e13-cebddab0d941 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:42.932063 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9f33f-df2f-49cd-8e13-cebddab0d941 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:42.942045 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 766/1513] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:28:43.793208 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c42874-1203-4899-a652-5d5058aec7e0 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:43.793801 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c42874-1203-4899-a652-5d5058aec7e0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:43.793801 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c42874-1203-4899-a652-5d5058aec7e0 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:43.795081 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 748/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:43.800462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baddb834-0e47-417e-975c-f84d3df112bb None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:43.800901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baddb834-0e47-417e-975c-f84d3df112bb None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:43.801019 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baddb834-0e47-417e-975c-f84d3df112bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:43.821053 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-baddb834-0e47-417e-975c-f84d3df112bb None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:43.822253 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-baddb834-0e47-417e-975c-f84d3df112bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:43.829834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 767/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:28:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:43.979236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c42874-1203-4899-a652-5d5058aec7e0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:43.994012 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c42874-1203-4899-a652-5d5058aec7e0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:43.999393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8597c-cb9c-4671-9efc-dbd7b6fbe426 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:43.999987 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8597c-cb9c-4671-9efc-dbd7b6fbe426 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:44.000276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8597c-cb9c-4671-9efc-dbd7b6fbe426 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:44.000554 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8597c-cb9c-4671-9efc-dbd7b6fbe426 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:44.008599 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d8597c-cb9c-4671-9efc-dbd7b6fbe426 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:44.010219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d8597c-cb9c-4671-9efc-dbd7b6fbe426 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:44.019736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 749/1516] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:45.821707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c610a5-a21f-47cf-9055-b2b028624788 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:45.822248 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c610a5-a21f-47cf-9055-b2b028624788 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:45.822248 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c610a5-a21f-47cf-9055-b2b028624788 None None] PATH_INFO: `/v3` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:45.823476 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 768/1517] 10.222.0.70 () {54 vars in 925 bytes} [Thu Sep 1 16:28:45 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:28:45.831499 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a29fad-2db3-4fb5-84a9-bb043fb656ac None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:45.831923 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a29fad-2db3-4fb5-84a9-bb043fb656ac None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:45.832088 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a29fad-2db3-4fb5-84a9-bb043fb656ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:45.892482 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a9a29fad-2db3-4fb5-84a9-bb043fb656ac None None] MFA Rules not processed for user `e12302cdb2854b0684179647a6dadf6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:45.893725 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a9a29fad-2db3-4fb5-84a9-bb043fb656ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:45.913380 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:45.969717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 750/1518] 10.222.0.70 () {58 vars in 1015 bytes} [Thu Sep 1 16:28:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:53.816144 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=56 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:55.221563 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18bab1-7152-4020-a979-c179650badef None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:55.222270 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18bab1-7152-4020-a979-c179650badef None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:55.222270 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18bab1-7152-4020-a979-c179650badef None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:55.223522 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 769/1519] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:55.228410 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe30ad-d1a1-4324-85c8-c1dc813d0f1d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:55.228841 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe30ad-d1a1-4324-85c8-c1dc813d0f1d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:55.228954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe30ad-d1a1-4324-85c8-c1dc813d0f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:55.251088 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b9fe30ad-d1a1-4324-85c8-c1dc813d0f1d None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:55.252799 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b9fe30ad-d1a1-4324-85c8-c1dc813d0f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:55.260788 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 751/1520] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:55.415280 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c18bab1-7152-4020-a979-c179650badef None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:55.429568 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c18bab1-7152-4020-a979-c179650badef None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:55.434906 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff3c3c-9bf7-4de9-b1db-0e2ad41301c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:55.435492 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff3c3c-9bf7-4de9-b1db-0e2ad41301c8 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:55.435767 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff3c3c-9bf7-4de9-b1db-0e2ad41301c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:55.436037 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff3c3c-9bf7-4de9-b1db-0e2ad41301c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:55.444355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff3c3c-9bf7-4de9-b1db-0e2ad41301c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:55.445904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff3c3c-9bf7-4de9-b1db-0e2ad41301c8 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:55.454941 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 770/1521] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:56.092643 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeca0a6-25c5-42f0-a6cb-c43d8ea04f8d None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:56.092881 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeca0a6-25c5-42f0-a6cb-c43d8ea04f8d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:56.093159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeca0a6-25c5-42f0-a6cb-c43d8ea04f8d None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:56.094541 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 752/1522] 10.222.0.58 () {54 vars in 961 bytes} [Thu Sep 1 16:28:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:56.100046 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007637dc-5991-435d-bd1e-587c3d30831b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:56.100329 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007637dc-5991-435d-bd1e-587c3d30831b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:56.100603 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007637dc-5991-435d-bd1e-587c3d30831b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:56.120470 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-007637dc-5991-435d-bd1e-587c3d30831b None None] MFA Rules not processed for user `e12302cdb2854b0684179647a6dadf6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:56.121651 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-007637dc-5991-435d-bd1e-587c3d30831b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:56.130211 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 771/1523] 10.222.0.58 () {58 vars in 1058 bytes} [Thu Sep 1 16:28:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:56.137347 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeca0a6-25c5-42f0-a6cb-c43d8ea04f8d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:56.150545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeca0a6-25c5-42f0-a6cb-c43d8ea04f8d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:56.155368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda0137-4714-4288-9869-ecce0f26f34e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12302cdb2854b0684179647a6dadf6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:56.155953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda0137-4714-4288-9869-ecce0f26f34e service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:56.156236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda0137-4714-4288-9869-ecce0f26f34e service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:56.156509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda0137-4714-4288-9869-ecce0f26f34e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:56.165254 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eda0137-4714-4288-9869-ecce0f26f34e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:56.166865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eda0137-4714-4288-9869-ecce0f26f34e service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:56.177158 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 753/1524] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:56.296024 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928cf74-7842-46b9-a5b0-4f3003171a2d None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:56.296592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928cf74-7842-46b9-a5b0-4f3003171a2d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:56.296592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928cf74-7842-46b9-a5b0-4f3003171a2d None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:56.297920 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 772/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:28:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:56.303505 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8fc2c9-d78c-4b2a-8c84-68aba28ff649 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:56.303965 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8fc2c9-d78c-4b2a-8c84-68aba28ff649 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:56.304069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8fc2c9-d78c-4b2a-8c84-68aba28ff649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:56.444397 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ce8fc2c9-d78c-4b2a-8c84-68aba28ff649 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:56.445595 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ce8fc2c9-d78c-4b2a-8c84-68aba28ff649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:56.453468 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 754/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:56.610475 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928cf74-7842-46b9-a5b0-4f3003171a2d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:56.624986 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928cf74-7842-46b9-a5b0-4f3003171a2d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:56.630397 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e8b514-d8da-46da-b263-2b33f8da8290 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:56.631040 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8b514-d8da-46da-b263-2b33f8da8290 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:56.631317 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8b514-d8da-46da-b263-2b33f8da8290 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:56.631585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8b514-d8da-46da-b263-2b33f8da8290 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:56.640209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e8b514-d8da-46da-b263-2b33f8da8290 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:56.641822 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e8b514-d8da-46da-b263-2b33f8da8290 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:56.652336 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 773/1527] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:29:00.675996 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb54650-78f1-4ab6-9904-d1104bb2d8fc None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:00.676569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb54650-78f1-4ab6-9904-d1104bb2d8fc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:00.677009 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb54650-78f1-4ab6-9904-d1104bb2d8fc None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:00.679482 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 755/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:29:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:29:00.685290 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19b842-b742-4c69-8bb2-91a7950ba6d7 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:00.685763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19b842-b742-4c69-8bb2-91a7950ba6d7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:00.685862 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19b842-b742-4c69-8bb2-91a7950ba6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:00.708977 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-9c19b842-b742-4c69-8bb2-91a7950ba6d7 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:00.710267 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9c19b842-b742-4c69-8bb2-91a7950ba6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:00.721146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 774/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:29:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:01.036075 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb54650-78f1-4ab6-9904-d1104bb2d8fc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:01.038416 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5eb54650-78f1-4ab6-9904-d1104bb2d8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:01.123587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb54650-78f1-4ab6-9904-d1104bb2d8fc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:01.127095 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159bb128-0cf4-4507-be8b-e7ee694f534c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:01.127694 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bb128-0cf4-4507-be8b-e7ee694f534c service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:01.127991 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bb128-0cf4-4507-be8b-e7ee694f534c service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:01.128243 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bb128-0cf4-4507-be8b-e7ee694f534c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:01.136865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159bb128-0cf4-4507-be8b-e7ee694f534c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:01.138305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159bb128-0cf4-4507-be8b-e7ee694f534c service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:01.148005 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 756/1530] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:29:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:29:03.043884 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02549f97-3ed7-4219-8cb5-c4b588bf47dd None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:03.044425 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02549f97-3ed7-4219-8cb5-c4b588bf47dd None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:03.044425 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02549f97-3ed7-4219-8cb5-c4b588bf47dd None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:03.045779 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 775/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:29:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:29:03.051792 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a90d3-0ef2-4678-984c-3717275c0dae None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:03.052211 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a90d3-0ef2-4678-984c-3717275c0dae None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:03.052320 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a90d3-0ef2-4678-984c-3717275c0dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:03.071411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5c0a90d3-0ef2-4678-984c-3717275c0dae None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:03.072590 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5c0a90d3-0ef2-4678-984c-3717275c0dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:03.080784 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 757/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:03.371012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02549f97-3ed7-4219-8cb5-c4b588bf47dd None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:03.381780 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02549f97-3ed7-4219-8cb5-c4b588bf47dd None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:03.385302 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea9092-e8e0-4691-82e4-89465ad8a314 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:03.385896 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea9092-e8e0-4691-82e4-89465ad8a314 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:03.386178 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea9092-e8e0-4691-82e4-89465ad8a314 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:03.386482 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea9092-e8e0-4691-82e4-89465ad8a314 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:03.395690 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea9092-e8e0-4691-82e4-89465ad8a314 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:03.397112 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea9092-e8e0-4691-82e4-89465ad8a314 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:03.407456 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 776/1533] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:29:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:29:06.849274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04cf40-b109-4ad4-8bd5-e40f38d1e27f None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:06.849853 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04cf40-b109-4ad4-8bd5-e40f38d1e27f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:06.849853 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04cf40-b109-4ad4-8bd5-e40f38d1e27f None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:06.851194 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 758/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:29:06.856380 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba986794-0c5d-4325-98de-7de23d563bb1 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:06.856854 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba986794-0c5d-4325-98de-7de23d563bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:06.856970 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba986794-0c5d-4325-98de-7de23d563bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:06.877698 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ba986794-0c5d-4325-98de-7de23d563bb1 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:06.878938 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ba986794-0c5d-4325-98de-7de23d563bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:06.887091 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 777/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:29:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:07.480148 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04cf40-b109-4ad4-8bd5-e40f38d1e27f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:07.481839 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-be04cf40-b109-4ad4-8bd5-e40f38d1e27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:07.547163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:07.562254 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04cf40-b109-4ad4-8bd5-e40f38d1e27f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:07.565839 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2be8d-06ef-470c-b5d4-722b46a4b7d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:07.566532 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2be8d-06ef-470c-b5d4-722b46a4b7d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:07.566762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2be8d-06ef-470c-b5d4-722b46a4b7d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:07.567035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2be8d-06ef-470c-b5d4-722b46a4b7d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:07.575918 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2be8d-06ef-470c-b5d4-722b46a4b7d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:07.577346 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2be8d-06ef-470c-b5d4-722b46a4b7d2 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:07.587903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 759/1536] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:29:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:29:09.512318 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64a5df-372d-4192-8a1f-7f5584aa79aa None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:09.512818 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64a5df-372d-4192-8a1f-7f5584aa79aa None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:09.512924 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64a5df-372d-4192-8a1f-7f5584aa79aa None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:09.514218 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 778/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:29:09.520178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b0afd-5699-4bed-8c8e-c171352fb81b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:09.520602 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b0afd-5699-4bed-8c8e-c171352fb81b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:09.520724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b0afd-5699-4bed-8c8e-c171352fb81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:09.540040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c00b0afd-5699-4bed-8c8e-c171352fb81b None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:09.541226 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c00b0afd-5699-4bed-8c8e-c171352fb81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:09.549003 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 760/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:10.100092 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64a5df-372d-4192-8a1f-7f5584aa79aa None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:10.101784 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1e64a5df-372d-4192-8a1f-7f5584aa79aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:10.114732 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64a5df-372d-4192-8a1f-7f5584aa79aa None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:10.118175 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f1352-eb6a-4348-829d-0c4f8455d044 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:10.118802 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f1352-eb6a-4348-829d-0c4f8455d044 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:10.119080 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f1352-eb6a-4348-829d-0c4f8455d044 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:10.119354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f1352-eb6a-4348-829d-0c4f8455d044 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:10.128055 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f1352-eb6a-4348-829d-0c4f8455d044 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:10.129462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f1352-eb6a-4348-829d-0c4f8455d044 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:10.139117 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 779/1539] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:29:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:29:19.534555 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=11 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:20.134971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=33 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:37.460074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4ecc2-6fc2-41c8-8b73-5c135d65ec7b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:37.460074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4ecc2-6fc2-41c8-8b73-5c135d65ec7b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:37.460074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4ecc2-6fc2-41c8-8b73-5c135d65ec7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:37.488953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4fb4ecc2-6fc2-41c8-8b73-5c135d65ec7b None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:37.490167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4fb4ecc2-6fc2-41c8-8b73-5c135d65ec7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:37.510170 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 761/1540] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:37.518995 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ecfa7-0d27-4e60-b8e5-7fb0125969fc None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:37.520623 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ecfa7-0d27-4e60-b8e5-7fb0125969fc None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:37.521147 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ecfa7-0d27-4e60-b8e5-7fb0125969fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:37.557540 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5b1ecfa7-0d27-4e60-b8e5-7fb0125969fc None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:37.559824 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5b1ecfa7-0d27-4e60-b8e5-7fb0125969fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:37.570056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 780/1541] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:37.579312 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb4ecc2-6fc2-41c8-8b73-5c135d65ec7b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:37.595769 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb4ecc2-6fc2-41c8-8b73-5c135d65ec7b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:37.606306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb872d-cf2e-4d2b-9a78-eb798438e609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:37.606947 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb872d-cf2e-4d2b-9a78-eb798438e609 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:37.607235 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb872d-cf2e-4d2b-9a78-eb798438e609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:37.607516 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb872d-cf2e-4d2b-9a78-eb798438e609 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:37.608208 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb872d-cf2e-4d2b-9a78-eb798438e609 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:30:37.608522 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb872d-cf2e-4d2b-9a78-eb798438e609 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:37.610136 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb872d-cf2e-4d2b-9a78-eb798438e609 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:37.641207 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 762/1542] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:30:37.649403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ecfa7-0d27-4e60-b8e5-7fb0125969fc None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:37.675373 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ecfa7-0d27-4e60-b8e5-7fb0125969fc None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:37.680389 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed42bf-63b4-4cb3-92a2-7e9cfb22ebe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:37.681010 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed42bf-63b4-4cb3-92a2-7e9cfb22ebe7 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:37.681291 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed42bf-63b4-4cb3-92a2-7e9cfb22ebe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:37.681565 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed42bf-63b4-4cb3-92a2-7e9cfb22ebe7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:37.682218 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed42bf-63b4-4cb3-92a2-7e9cfb22ebe7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:37.684188 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed42bf-63b4-4cb3-92a2-7e9cfb22ebe7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:37.769696 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 781/1543] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:30:37 2022] POST /identity/v3/projects => generated 401 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:30:37.779337 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb872d-cf2e-4d2b-9a78-eb798438e609 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:37.803064 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb872d-cf2e-4d2b-9a78-eb798438e609 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:37.925203 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa1b37-4915-46aa-9abd-4e1a047c6745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:37.925844 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa1b37-4915-46aa-9abd-4e1a047c6745 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:37.926131 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa1b37-4915-46aa-9abd-4e1a047c6745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:37.926437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa1b37-4915-46aa-9abd-4e1a047c6745 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:37.927096 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fa1b37-4915-46aa-9abd-4e1a047c6745 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:37.929114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fa1b37-4915-46aa-9abd-4e1a047c6745 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:37.987233 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 763/1544] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:30:37 2022] POST /identity/v3/users => generated 301 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:30:37.996841 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed42bf-63b4-4cb3-92a2-7e9cfb22ebe7 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:38.015891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed42bf-63b4-4cb3-92a2-7e9cfb22ebe7 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:38.029367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2a83a-6d84-4053-b199-0f7fab699626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:38.029923 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2a83a-6d84-4053-b199-0f7fab699626 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:38.030349 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2a83a-6d84-4053-b199-0f7fab699626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:38.030680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2a83a-6d84-4053-b199-0f7fab699626 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:38.031363 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e2a83a-6d84-4053-b199-0f7fab699626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:38.033256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e2a83a-6d84-4053-b199-0f7fab699626 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:38.057701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 782/1545] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:30:38.072328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa1b37-4915-46aa-9abd-4e1a047c6745 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:38.100702 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa1b37-4915-46aa-9abd-4e1a047c6745 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:38.113974 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89354e79-8a01-460c-ba85-34b63ed63e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:38.114667 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89354e79-8a01-460c-ba85-34b63ed63e43 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:38.114969 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89354e79-8a01-460c-ba85-34b63ed63e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:38.115272 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89354e79-8a01-460c-ba85-34b63ed63e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:38.115946 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89354e79-8a01-460c-ba85-34b63ed63e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:38.118506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89354e79-8a01-460c-ba85-34b63ed63e43 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:38.146104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 764/1546] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:30:38.150003 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2a83a-6d84-4053-b199-0f7fab699626 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:38.177458 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2a83a-6d84-4053-b199-0f7fab699626 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:38.198246 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8690850-2c38-45d7-9b06-dc8fd5c3dfdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:38.198246 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8690850-2c38-45d7-9b06-dc8fd5c3dfdf admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:38.198246 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8690850-2c38-45d7-9b06-dc8fd5c3dfdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:38.198246 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8690850-2c38-45d7-9b06-dc8fd5c3dfdf admin admin] PATH_INFO: `/v3/projects/f3cd1605b6ee4e52ab0c2edab29acbe3/users/0afe9571b2304659b7be359fbda93d2c/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:38.235290 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8690850-2c38-45d7-9b06-dc8fd5c3dfdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3cd1605b6ee4e52ab0c2edab29acbe3, user_id=0afe9571b2304659b7be359fbda93d2c, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:38.238901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8690850-2c38-45d7-9b06-dc8fd5c3dfdf admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:38.276008 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 783/1547] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:30:38 2022] PUT /identity/v3/projects/f3cd1605b6ee4e52ab0c2edab29acbe3/users/0afe9571b2304659b7be359fbda93d2c/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:30:38.291937 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ddd46-ca0d-404c-bc97-1ea8112e2150 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:38.292139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ddd46-ca0d-404c-bc97-1ea8112e2150 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:38.292400 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ddd46-ca0d-404c-bc97-1ea8112e2150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:38.363063 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-157ddd46-ca0d-404c-bc97-1ea8112e2150 None None] MFA Rules not processed for user `0afe9571b2304659b7be359fbda93d2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:38.364370 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-157ddd46-ca0d-404c-bc97-1ea8112e2150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:38.443829 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3abd9d-7dd5-4bb5-9162-2bceea344c82 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:38.444198 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3abd9d-7dd5-4bb5-9162-2bceea344c82 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:38.444387 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3abd9d-7dd5-4bb5-9162-2bceea344c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:38.503538 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-fc3abd9d-7dd5-4bb5-9162-2bceea344c82 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:38.504771 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-fc3abd9d-7dd5-4bb5-9162-2bceea344c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:38.510117 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 765/1548] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:38 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:38.514298 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27b4eb-3bd4-4bb6-ab06-c35216674961 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:38.518146 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27b4eb-3bd4-4bb6-ab06-c35216674961 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:38.518146 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27b4eb-3bd4-4bb6-ab06-c35216674961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:38.561223 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9e27b4eb-3bd4-4bb6-ab06-c35216674961 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:38.567550 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9e27b4eb-3bd4-4bb6-ab06-c35216674961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:38.662615 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 784/1549] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:38.664889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c11d7c-4aa6-4bdf-bbde-11a11293401e None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:38.664889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c11d7c-4aa6-4bdf-bbde-11a11293401e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:38.664889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c11d7c-4aa6-4bdf-bbde-11a11293401e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:38.703002 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a3c11d7c-4aa6-4bdf-bbde-11a11293401e None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:38.708856 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a3c11d7c-4aa6-4bdf-bbde-11a11293401e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:38.722569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:38.722876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 785/1550] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:38.724120 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c11d7c-4aa6-4bdf-bbde-11a11293401e None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:38.736525 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c11d7c-4aa6-4bdf-bbde-11a11293401e None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:38.742695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f050067-2774-4d7c-a502-b536384eae2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:38.743626 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f050067-2774-4d7c-a502-b536384eae2d admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:38.744204 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f050067-2774-4d7c-a502-b536384eae2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:38.744842 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f050067-2774-4d7c-a502-b536384eae2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:38.745830 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f050067-2774-4d7c-a502-b536384eae2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:38.747318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 766/1551] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:38.748639 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f050067-2774-4d7c-a502-b536384eae2d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:38.748994 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07799630-d22d-41db-b939-ae75fa978d96 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:38.749470 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07799630-d22d-41db-b939-ae75fa978d96 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:38.749761 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07799630-d22d-41db-b939-ae75fa978d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:38.775721 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-07799630-d22d-41db-b939-ae75fa978d96 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:38.778851 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-07799630-d22d-41db-b939-ae75fa978d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:38.791397 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 767/1552] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:38.793120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1404d3-997d-4778-b57b-1511e753a049 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:38.793648 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1404d3-997d-4778-b57b-1511e753a049 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:38.794041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1404d3-997d-4778-b57b-1511e753a049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:38.797867 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 786/1553] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:30:38 2022] POST /identity/v3/projects => generated 399 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:30:38.797867 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4fa86d-393d-4111-b394-085697f3a832 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:38.797867 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4fa86d-393d-4111-b394-085697f3a832 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:38.797867 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4fa86d-393d-4111-b394-085697f3a832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:38.820318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7a1404d3-997d-4778-b57b-1511e753a049 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:38.822579 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7a1404d3-997d-4778-b57b-1511e753a049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:38.828391 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-de4fa86d-393d-4111-b394-085697f3a832 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:38.829598 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-de4fa86d-393d-4111-b394-085697f3a832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:38.891424 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 768/1554] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:38.892740 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:38.892898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d55f2e-7a7a-4a48-b955-00a3cae8f972 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:38.893095 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d55f2e-7a7a-4a48-b955-00a3cae8f972 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:38.893321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d55f2e-7a7a-4a48-b955-00a3cae8f972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:38.911159 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 787/1555] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:38.913238 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cd194-dbf7-488f-a7ea-ec501208a42b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:38.913639 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cd194-dbf7-488f-a7ea-ec501208a42b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:38.914237 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cd194-dbf7-488f-a7ea-ec501208a42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:38.915612 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-90d55f2e-7a7a-4a48-b955-00a3cae8f972 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:38.916857 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-90d55f2e-7a7a-4a48-b955-00a3cae8f972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:38.926850 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 769/1556] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:38.928629 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98179f3a-e73d-4096-ba97-7f621c5e357c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:38.929092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98179f3a-e73d-4096-ba97-7f621c5e357c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:38.929534 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98179f3a-e73d-4096-ba97-7f621c5e357c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:38.940405 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d60cd194-dbf7-488f-a7ea-ec501208a42b None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:38.941713 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d60cd194-dbf7-488f-a7ea-ec501208a42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:38.950081 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-98179f3a-e73d-4096-ba97-7f621c5e357c None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:38.950623 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 788/1557] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:38.951328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-98179f3a-e73d-4096-ba97-7f621c5e357c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:38.951575 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60cd194-dbf7-488f-a7ea-ec501208a42b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:38.962966 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60cd194-dbf7-488f-a7ea-ec501208a42b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:38.966766 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 770/1558] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:38.968768 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc5b04-caa1-4c20-9e09-2f786e4c29d4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:38.969222 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc5b04-caa1-4c20-9e09-2f786e4c29d4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:38.969581 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc5b04-caa1-4c20-9e09-2f786e4c29d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:38.972050 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d1605-ed1a-4423-9e54-cf90e6f0a89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:38.972804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d1605-ed1a-4423-9e54-cf90e6f0a89c admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:38.973088 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d1605-ed1a-4423-9e54-cf90e6f0a89c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:38.973479 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d1605-ed1a-4423-9e54-cf90e6f0a89c admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:38.974281 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d1605-ed1a-4423-9e54-cf90e6f0a89c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:38.975304 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: WARNING py.warnings [None req-969d1605-ed1a-4423-9e54-cf90e6f0a89c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:30:38.975304 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: warnings.warn(msg) Sep 01 16:30:38.975304 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: Sep 01 16:30:38.976518 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d1605-ed1a-4423-9e54-cf90e6f0a89c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:38.991318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8bdc5b04-caa1-4c20-9e09-2f786e4c29d4 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:38.993543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8bdc5b04-caa1-4c20-9e09-2f786e4c29d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:39.002916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 789/1559] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:30:38 2022] POST /identity/v3/users => generated 306 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:30:39.003085 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 771/1560] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:39.004278 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7bda1-38c8-4262-9a8e-03ed9d107ef6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:39.004482 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b5266-149b-4287-ac20-ef07cc31cfb7 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:39.004602 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7bda1-38c8-4262-9a8e-03ed9d107ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:39.004750 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b5266-149b-4287-ac20-ef07cc31cfb7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:39.004849 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7bda1-38c8-4262-9a8e-03ed9d107ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:39.005022 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b5266-149b-4287-ac20-ef07cc31cfb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:39.026052 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5de7bda1-38c8-4262-9a8e-03ed9d107ef6 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:39.027294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5de7bda1-38c8-4262-9a8e-03ed9d107ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:39.031658 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e78b5266-149b-4287-ac20-ef07cc31cfb7 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:39.032943 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e78b5266-149b-4287-ac20-ef07cc31cfb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:39.036182 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 772/1561] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:39.043083 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 790/1562] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:39.044332 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b5266-149b-4287-ac20-ef07cc31cfb7 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:39.048319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de7bda1-38c8-4262-9a8e-03ed9d107ef6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:39.055547 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b5266-149b-4287-ac20-ef07cc31cfb7 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:39.060092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de7bda1-38c8-4262-9a8e-03ed9d107ef6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:39.060859 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f4262-bf7a-4216-8ca0-0cd29f9307fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:39.061682 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f4262-bf7a-4216-8ca0-0cd29f9307fa admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:39.061962 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f4262-bf7a-4216-8ca0-0cd29f9307fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:39.062265 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f4262-bf7a-4216-8ca0-0cd29f9307fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:39.063082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f4262-bf7a-4216-8ca0-0cd29f9307fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:30:39.063393 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f4262-bf7a-4216-8ca0-0cd29f9307fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:39.064420 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: WARNING py.warnings [None req-cc7f4262-bf7a-4216-8ca0-0cd29f9307fa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:30:39.064420 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: warnings.warn(msg) Sep 01 16:30:39.064420 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: Sep 01 16:30:39.065318 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f4262-bf7a-4216-8ca0-0cd29f9307fa admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:39.065459 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7057b7-4fd7-4f65-8874-4e3849d87e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:39.066359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7057b7-4fd7-4f65-8874-4e3849d87e82 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:39.066555 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7057b7-4fd7-4f65-8874-4e3849d87e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:39.066987 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7057b7-4fd7-4f65-8874-4e3849d87e82 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:39.067798 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7057b7-4fd7-4f65-8874-4e3849d87e82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:30:39.068159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7057b7-4fd7-4f65-8874-4e3849d87e82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:39.069866 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7057b7-4fd7-4f65-8874-4e3849d87e82 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:39.081020 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 791/1563] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:30:39.082152 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99fe35-af72-443a-9515-3c971783b30a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:39.082480 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99fe35-af72-443a-9515-3c971783b30a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:39.082757 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99fe35-af72-443a-9515-3c971783b30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:39.090333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 773/1564] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:30:39.090333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7057b7-4fd7-4f65-8874-4e3849d87e82 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:39.106584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7057b7-4fd7-4f65-8874-4e3849d87e82 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:39.108233 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-6e99fe35-af72-443a-9515-3c971783b30a None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:39.109547 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6e99fe35-af72-443a-9515-3c971783b30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:39.111592 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9113b-67a2-4c9e-b4fa-2ac9038a6f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:39.112280 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9113b-67a2-4c9e-b4fa-2ac9038a6f75 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:39.112572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9113b-67a2-4c9e-b4fa-2ac9038a6f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:39.112947 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9113b-67a2-4c9e-b4fa-2ac9038a6f75 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:39.113692 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a9113b-67a2-4c9e-b4fa-2ac9038a6f75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:30:39.114033 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a9113b-67a2-4c9e-b4fa-2ac9038a6f75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:39.115669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a9113b-67a2-4c9e-b4fa-2ac9038a6f75 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:39.118281 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 792/1565] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:39.119086 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e99fe35-af72-443a-9515-3c971783b30a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:39.130880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e99fe35-af72-443a-9515-3c971783b30a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:39.134353 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 774/1566] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:30:39.134353 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9113b-67a2-4c9e-b4fa-2ac9038a6f75 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:39.137976 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558d5ab-03e1-4a50-96e9-eb8b3ae9b0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:39.139100 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558d5ab-03e1-4a50-96e9-eb8b3ae9b0d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:39.139100 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558d5ab-03e1-4a50-96e9-eb8b3ae9b0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:39.141165 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558d5ab-03e1-4a50-96e9-eb8b3ae9b0d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:39.141165 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558d5ab-03e1-4a50-96e9-eb8b3ae9b0d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:39.142954 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558d5ab-03e1-4a50-96e9-eb8b3ae9b0d1 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:39.145874 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9113b-67a2-4c9e-b4fa-2ac9038a6f75 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:39.151129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cea762-656d-4178-80d1-3957bf1ee97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:39.151734 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cea762-656d-4178-80d1-3957bf1ee97a admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:39.152018 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cea762-656d-4178-80d1-3957bf1ee97a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:39.152293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cea762-656d-4178-80d1-3957bf1ee97a admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:39.152943 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cea762-656d-4178-80d1-3957bf1ee97a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:30:39.153248 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cea762-656d-4178-80d1-3957bf1ee97a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:39.156029 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 793/1567] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:30:39.156168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cea762-656d-4178-80d1-3957bf1ee97a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:39.157005 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75c201-e53f-44af-b638-91ab37f90fc7 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:39.157477 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75c201-e53f-44af-b638-91ab37f90fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:39.157939 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75c201-e53f-44af-b638-91ab37f90fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:39.169024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 775/1568] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:30:39.169877 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cea762-656d-4178-80d1-3957bf1ee97a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:39.180801 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-3c75c201-e53f-44af-b638-91ab37f90fc7 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:39.182440 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3c75c201-e53f-44af-b638-91ab37f90fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:39.186740 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cea762-656d-4178-80d1-3957bf1ee97a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:39.191246 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 794/1569] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:39.192151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75c201-e53f-44af-b638-91ab37f90fc7 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:39.196369 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0dc45-0297-42eb-8f16-001841f6dc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:39.197278 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0dc45-0297-42eb-8f16-001841f6dc60 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:39.197515 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0dc45-0297-42eb-8f16-001841f6dc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:39.198100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0dc45-0297-42eb-8f16-001841f6dc60 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:39.198863 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0dc45-0297-42eb-8f16-001841f6dc60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:30:39.199237 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0dc45-0297-42eb-8f16-001841f6dc60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:39.200945 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0dc45-0297-42eb-8f16-001841f6dc60 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:39.203575 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75c201-e53f-44af-b638-91ab37f90fc7 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:39.208788 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784a448-4e34-4f4a-af59-03e4f85bb31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:39.209395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784a448-4e34-4f4a-af59-03e4f85bb31d admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:39.210517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784a448-4e34-4f4a-af59-03e4f85bb31d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:39.210833 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784a448-4e34-4f4a-af59-03e4f85bb31d admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:39.211502 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e784a448-4e34-4f4a-af59-03e4f85bb31d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:39.213449 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e784a448-4e34-4f4a-af59-03e4f85bb31d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:39.222483 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 776/1570] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:30:39.222483 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0dc45-0297-42eb-8f16-001841f6dc60 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:39.229957 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0dc45-0297-42eb-8f16-001841f6dc60 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:39.236197 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd95de-cf77-4732-83f6-d17c864bbc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:39.236197 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd95de-cf77-4732-83f6-d17c864bbc6a admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:39.236478 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd95de-cf77-4732-83f6-d17c864bbc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:39.236577 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd95de-cf77-4732-83f6-d17c864bbc6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:39.237162 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd95de-cf77-4732-83f6-d17c864bbc6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:39.238387 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-10bd95de-cf77-4732-83f6-d17c864bbc6a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:30:39.238387 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Sep 01 16:30:39.238387 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: Sep 01 16:30:39.239761 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd95de-cf77-4732-83f6-d17c864bbc6a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:39.254279 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 795/1571] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:30:39 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:30:39.254279 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784a448-4e34-4f4a-af59-03e4f85bb31d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:39.273377 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 777/1572] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:30:39 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:30:39.274292 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd95de-cf77-4732-83f6-d17c864bbc6a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:39.322125 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784a448-4e34-4f4a-af59-03e4f85bb31d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:39.323678 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:39.346383 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd95de-cf77-4732-83f6-d17c864bbc6a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:39.352804 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55511b04-4c97-4d1c-a9b1-7c347cf965c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:39.353495 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55511b04-4c97-4d1c-a9b1-7c347cf965c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:39.353788 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55511b04-4c97-4d1c-a9b1-7c347cf965c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:39.354095 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55511b04-4c97-4d1c-a9b1-7c347cf965c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:39.354735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55511b04-4c97-4d1c-a9b1-7c347cf965c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:39.356645 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55511b04-4c97-4d1c-a9b1-7c347cf965c8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:39.384542 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0063d8-cce1-4c48-9c3f-0f189b1c91fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:39.385156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0063d8-cce1-4c48-9c3f-0f189b1c91fa admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:39.385442 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0063d8-cce1-4c48-9c3f-0f189b1c91fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:39.385713 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0063d8-cce1-4c48-9c3f-0f189b1c91fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:39.386431 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0063d8-cce1-4c48-9c3f-0f189b1c91fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:30:39.386740 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0063d8-cce1-4c48-9c3f-0f189b1c91fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:39.388414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0063d8-cce1-4c48-9c3f-0f189b1c91fa admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:39.390362 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 778/1573] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:30:39 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:30:39.391258 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55511b04-4c97-4d1c-a9b1-7c347cf965c8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:39.402793 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 796/1574] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:30:39.403598 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0063d8-cce1-4c48-9c3f-0f189b1c91fa admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:39.416066 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0063d8-cce1-4c48-9c3f-0f189b1c91fa admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:39.426822 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:39.477659 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55511b04-4c97-4d1c-a9b1-7c347cf965c8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:39.482527 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3d6de-fbab-4fee-9af2-eb2ffdcc1e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:39.483330 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3d6de-fbab-4fee-9af2-eb2ffdcc1e56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:39.484909 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3fb8f6-f5cc-4f63-a9b0-7e486b729c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:39.485582 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fb8f6-f5cc-4f63-a9b0-7e486b729c6b admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:39.485922 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fb8f6-f5cc-4f63-a9b0-7e486b729c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:39.486385 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fb8f6-f5cc-4f63-a9b0-7e486b729c6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:39.486996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3fb8f6-f5cc-4f63-a9b0-7e486b729c6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:39.488206 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3d6de-fbab-4fee-9af2-eb2ffdcc1e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:39.488492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3d6de-fbab-4fee-9af2-eb2ffdcc1e56 admin admin] PATH_INFO: `/v3/projects/55b0149dfe5146ee955a0bb80784e064/users/cc722bd6c1ed489c8be32e20b599249c/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:39.489338 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3fb8f6-f5cc-4f63-a9b0-7e486b729c6b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:39.505961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e3d6de-fbab-4fee-9af2-eb2ffdcc1e56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55b0149dfe5146ee955a0bb80784e064, user_id=cc722bd6c1ed489c8be32e20b599249c, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:39.507208 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-e5e3d6de-fbab-4fee-9af2-eb2ffdcc1e56 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:30:39.507208 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Sep 01 16:30:39.507208 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: Sep 01 16:30:39.509190 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e3d6de-fbab-4fee-9af2-eb2ffdcc1e56 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:39.524206 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 797/1575] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:30:39 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:30:39.525155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3fb8f6-f5cc-4f63-a9b0-7e486b729c6b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:39.530330 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 779/1576] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:30:39 2022] PUT /identity/v3/projects/55b0149dfe5146ee955a0bb80784e064/users/cc722bd6c1ed489c8be32e20b599249c/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:30:39.530330 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3d6de-fbab-4fee-9af2-eb2ffdcc1e56 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:39.595412 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3fb8f6-f5cc-4f63-a9b0-7e486b729c6b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:39.600648 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae687c5c-c482-444d-802e-397514269400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:39.601266 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae687c5c-c482-444d-802e-397514269400 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:39.601626 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae687c5c-c482-444d-802e-397514269400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:39.601984 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae687c5c-c482-444d-802e-397514269400 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:39.602770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae687c5c-c482-444d-802e-397514269400 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:30:39.603072 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae687c5c-c482-444d-802e-397514269400 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:39.604793 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae687c5c-c482-444d-802e-397514269400 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:39.608265 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3d6de-fbab-4fee-9af2-eb2ffdcc1e56 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:39.612979 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190012f1-d3d0-4fd8-996f-278496282568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:39.613576 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190012f1-d3d0-4fd8-996f-278496282568 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:39.613862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190012f1-d3d0-4fd8-996f-278496282568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:39.614133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190012f1-d3d0-4fd8-996f-278496282568 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:39.614820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190012f1-d3d0-4fd8-996f-278496282568 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:39.616727 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190012f1-d3d0-4fd8-996f-278496282568 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:39.617393 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 798/1577] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:30:39.619530 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae687c5c-c482-444d-802e-397514269400 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:39.633087 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae687c5c-c482-444d-802e-397514269400 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:39.638018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1d27e-9422-40f5-8b50-43fd64e13a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:39.639137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1d27e-9422-40f5-8b50-43fd64e13a3a admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:39.639137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1d27e-9422-40f5-8b50-43fd64e13a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:39.640128 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1d27e-9422-40f5-8b50-43fd64e13a3a admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:39.640815 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b1d27e-9422-40f5-8b50-43fd64e13a3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:39.643096 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b1d27e-9422-40f5-8b50-43fd64e13a3a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:39.651219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 780/1578] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:30:39 2022] POST /identity/v3/projects => generated 377 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:30:39.651219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190012f1-d3d0-4fd8-996f-278496282568 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:39.667639 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 799/1579] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:30:39 2022] POST /identity/v3/users => generated 286 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:30:39.668586 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1d27e-9422-40f5-8b50-43fd64e13a3a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:39.681614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1d27e-9422-40f5-8b50-43fd64e13a3a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:39.700317 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:39.733432 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190012f1-d3d0-4fd8-996f-278496282568 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:39.738779 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63479717-efa8-48dc-83b2-4abf4e26e204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:39.739398 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63479717-efa8-48dc-83b2-4abf4e26e204 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:39.739688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63479717-efa8-48dc-83b2-4abf4e26e204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:39.740155 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63479717-efa8-48dc-83b2-4abf4e26e204 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:39.740797 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63479717-efa8-48dc-83b2-4abf4e26e204 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:39.742859 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63479717-efa8-48dc-83b2-4abf4e26e204 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:39.755942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0ecf5-f5be-41a0-8e8e-9a51a5ea9895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:39.756566 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0ecf5-f5be-41a0-8e8e-9a51a5ea9895 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:39.756855 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0ecf5-f5be-41a0-8e8e-9a51a5ea9895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:39.757672 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0ecf5-f5be-41a0-8e8e-9a51a5ea9895 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:39.758461 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e0ecf5-f5be-41a0-8e8e-9a51a5ea9895 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:39.763123 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e0ecf5-f5be-41a0-8e8e-9a51a5ea9895 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:39.770462 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 781/1580] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:30:39 2022] POST /identity/v3/users => generated 282 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:30:39.771380 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63479717-efa8-48dc-83b2-4abf4e26e204 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:39.785428 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63479717-efa8-48dc-83b2-4abf4e26e204 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:39.791660 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76baee65-39db-47be-b99c-6c15d26f916c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:39.792274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76baee65-39db-47be-b99c-6c15d26f916c admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:39.792580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76baee65-39db-47be-b99c-6c15d26f916c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:39.792856 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76baee65-39db-47be-b99c-6c15d26f916c admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:39.793494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76baee65-39db-47be-b99c-6c15d26f916c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:39.795439 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76baee65-39db-47be-b99c-6c15d26f916c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:39.797793 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 800/1581] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:30:39 2022] POST /identity/v3/users => generated 293 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:30:39.798799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0ecf5-f5be-41a0-8e8e-9a51a5ea9895 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:39.804585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:39.813321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0ecf5-f5be-41a0-8e8e-9a51a5ea9895 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:39.820182 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306920bc-f142-4e0a-9be8-0bdbb229705c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:39.821334 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306920bc-f142-4e0a-9be8-0bdbb229705c admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:39.821630 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306920bc-f142-4e0a-9be8-0bdbb229705c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:39.821909 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306920bc-f142-4e0a-9be8-0bdbb229705c admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:39.822728 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306920bc-f142-4e0a-9be8-0bdbb229705c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:39.825032 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306920bc-f142-4e0a-9be8-0bdbb229705c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:39.828052 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 782/1582] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:30:39 2022] POST /identity/v3/projects => generated 393 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:30:39.828897 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76baee65-39db-47be-b99c-6c15d26f916c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:39.849425 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 801/1583] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:30:39 2022] POST /identity/v3/users => generated 288 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:30:39.850778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306920bc-f142-4e0a-9be8-0bdbb229705c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:39.861347 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306920bc-f142-4e0a-9be8-0bdbb229705c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:39.901430 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76baee65-39db-47be-b99c-6c15d26f916c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:39.908843 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047af70-2602-483c-a851-cb167d12c728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:39.909511 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047af70-2602-483c-a851-cb167d12c728 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:39.910245 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047af70-2602-483c-a851-cb167d12c728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:39.910455 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047af70-2602-483c-a851-cb167d12c728 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:39.911115 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047af70-2602-483c-a851-cb167d12c728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:39.913008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047af70-2602-483c-a851-cb167d12c728 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:39.924113 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 783/1584] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:30:39.924775 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047af70-2602-483c-a851-cb167d12c728 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:39.928693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d21a7b6-98d8-44a8-b782-fce67dcec5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:39.929387 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21a7b6-98d8-44a8-b782-fce67dcec5f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:39.929730 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21a7b6-98d8-44a8-b782-fce67dcec5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:39.930070 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21a7b6-98d8-44a8-b782-fce67dcec5f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:39.930820 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d21a7b6-98d8-44a8-b782-fce67dcec5f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:39.932834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d21a7b6-98d8-44a8-b782-fce67dcec5f7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:39.938514 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047af70-2602-483c-a851-cb167d12c728 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:39.943125 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726b596-ecf1-4b1e-baf0-8cad067e7b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:39.943760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726b596-ecf1-4b1e-baf0-8cad067e7b91 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:39.944041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726b596-ecf1-4b1e-baf0-8cad067e7b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:39.944312 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726b596-ecf1-4b1e-baf0-8cad067e7b91 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:39.945006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7726b596-ecf1-4b1e-baf0-8cad067e7b91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:39.946927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7726b596-ecf1-4b1e-baf0-8cad067e7b91 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:39.966147 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 802/1585] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:30:39 2022] POST /identity/v3/projects => generated 389 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:30:39.967408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d21a7b6-98d8-44a8-b782-fce67dcec5f7 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:39.977754 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 784/1586] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:30:39 2022] POST /identity/v3/users => generated 289 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:30:39.977754 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726b596-ecf1-4b1e-baf0-8cad067e7b91 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:39.990188 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726b596-ecf1-4b1e-baf0-8cad067e7b91 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:40.049048 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d21a7b6-98d8-44a8-b782-fce67dcec5f7 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:40.054238 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e1fec-17f0-4f3d-a040-30ff0c8d558b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:40.054825 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e1fec-17f0-4f3d-a040-30ff0c8d558b admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:40.055105 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e1fec-17f0-4f3d-a040-30ff0c8d558b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:40.055380 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e1fec-17f0-4f3d-a040-30ff0c8d558b admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:40.056039 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e1fec-17f0-4f3d-a040-30ff0c8d558b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:40.057683 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e1fec-17f0-4f3d-a040-30ff0c8d558b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:40.060880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c88ea3-9309-41dd-b563-e8e46ba18574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:40.062046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c88ea3-9309-41dd-b563-e8e46ba18574 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:40.062380 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c88ea3-9309-41dd-b563-e8e46ba18574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:40.062708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c88ea3-9309-41dd-b563-e8e46ba18574 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:40.063359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c88ea3-9309-41dd-b563-e8e46ba18574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:40.067300 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c88ea3-9309-41dd-b563-e8e46ba18574 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:40.069558 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 803/1587] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:30:40.070406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e1fec-17f0-4f3d-a040-30ff0c8d558b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:40.080706 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e1fec-17f0-4f3d-a040-30ff0c8d558b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:40.084484 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 785/1588] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:30:40.084484 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c88ea3-9309-41dd-b563-e8e46ba18574 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:40.085823 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9fcf1-7cf5-429d-9503-f5984ac99c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:40.086554 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9fcf1-7cf5-429d-9503-f5984ac99c8a admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:40.086834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9fcf1-7cf5-429d-9503-f5984ac99c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:40.087149 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9fcf1-7cf5-429d-9503-f5984ac99c8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:40.088366 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9fcf1-7cf5-429d-9503-f5984ac99c8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:40.089396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9fcf1-7cf5-429d-9503-f5984ac99c8a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:40.094293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c88ea3-9309-41dd-b563-e8e46ba18574 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:40.100466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 804/1589] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:30:40.101349 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9fcf1-7cf5-429d-9503-f5984ac99c8a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:40.101477 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b2d23-ef52-401c-8ed7-8e266ce76828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:40.102065 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b2d23-ef52-401c-8ed7-8e266ce76828 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:40.102385 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b2d23-ef52-401c-8ed7-8e266ce76828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:40.102675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b2d23-ef52-401c-8ed7-8e266ce76828 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:40.103288 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b2d23-ef52-401c-8ed7-8e266ce76828 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:40.105696 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b2d23-ef52-401c-8ed7-8e266ce76828 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:40.111991 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9fcf1-7cf5-429d-9503-f5984ac99c8a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:40.117817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663cee4-0ab5-4add-84e9-c443ff2816ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:40.118610 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663cee4-0ab5-4add-84e9-c443ff2816ea admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:40.119110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663cee4-0ab5-4add-84e9-c443ff2816ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:40.119536 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663cee4-0ab5-4add-84e9-c443ff2816ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:40.120416 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1663cee4-0ab5-4add-84e9-c443ff2816ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:40.122340 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1663cee4-0ab5-4add-84e9-c443ff2816ea admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:40.126782 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:40.134910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 805/1590] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:30:40.135738 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663cee4-0ab5-4add-84e9-c443ff2816ea admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:40.138848 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 786/1591] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:30:40 2022] POST /identity/v3/users => generated 297 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:30:40.139775 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b2d23-ef52-401c-8ed7-8e266ce76828 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:40.146588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663cee4-0ab5-4add-84e9-c443ff2816ea admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:40.151563 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b2d23-ef52-401c-8ed7-8e266ce76828 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:40.152196 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea3598-9229-4da5-84cc-f089fa092a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:40.152837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea3598-9229-4da5-84cc-f089fa092a08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:40.153124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea3598-9229-4da5-84cc-f089fa092a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:40.153396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea3598-9229-4da5-84cc-f089fa092a08 admin admin] PATH_INFO: `/v3/projects/55b0149dfe5146ee955a0bb80784e064/users/cc722bd6c1ed489c8be32e20b599249c/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:40.156091 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea3598-9229-4da5-84cc-f089fa092a08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55b0149dfe5146ee955a0bb80784e064, user_id=cc722bd6c1ed489c8be32e20b599249c, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:40.157310 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6d89e-f524-402a-92eb-98b71be250e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:40.157917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6d89e-f524-402a-92eb-98b71be250e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:40.158216 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6d89e-f524-402a-92eb-98b71be250e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:40.158556 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6d89e-f524-402a-92eb-98b71be250e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:40.158845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea3598-9229-4da5-84cc-f089fa092a08 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:40.159247 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6d89e-f524-402a-92eb-98b71be250e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:40.161156 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6d89e-f524-402a-92eb-98b71be250e5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:40.171357 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 806/1592] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:30:40 2022] PUT /identity/v3/projects/55b0149dfe5146ee955a0bb80784e064/users/cc722bd6c1ed489c8be32e20b599249c/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:30:40.172109 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea3598-9229-4da5-84cc-f089fa092a08 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:40.193435 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 787/1593] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:30:40 2022] POST /identity/v3/users => generated 295 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:30:40.195550 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6d89e-f524-402a-92eb-98b71be250e5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:40.208004 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6d89e-f524-402a-92eb-98b71be250e5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:40.217020 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:40.248946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea3598-9229-4da5-84cc-f089fa092a08 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:40.254688 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f73e6b-4e75-479d-adac-4259d58323f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:40.255730 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f73e6b-4e75-479d-adac-4259d58323f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:40.255868 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f73e6b-4e75-479d-adac-4259d58323f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:40.256157 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f73e6b-4e75-479d-adac-4259d58323f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:40.257061 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f73e6b-4e75-479d-adac-4259d58323f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:40.258669 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f73e6b-4e75-479d-adac-4259d58323f5 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:40.271266 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 807/1594] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:30:40.271984 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f73e6b-4e75-479d-adac-4259d58323f5 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:40.273819 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c444cc33-7b34-4778-a385-084142446405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:40.274774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444cc33-7b34-4778-a385-084142446405 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:40.275349 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444cc33-7b34-4778-a385-084142446405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:40.275631 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444cc33-7b34-4778-a385-084142446405 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:40.276284 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c444cc33-7b34-4778-a385-084142446405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:40.277898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c444cc33-7b34-4778-a385-084142446405 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:40.290456 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f73e6b-4e75-479d-adac-4259d58323f5 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:40.290663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 788/1595] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:30:40.290663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c444cc33-7b34-4778-a385-084142446405 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:40.297045 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398872c-333e-4a2c-94c4-6f4184e390b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:40.297730 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398872c-333e-4a2c-94c4-6f4184e390b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:40.298075 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398872c-333e-4a2c-94c4-6f4184e390b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:40.298479 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398872c-333e-4a2c-94c4-6f4184e390b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:40.299195 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398872c-333e-4a2c-94c4-6f4184e390b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:40.302362 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398872c-333e-4a2c-94c4-6f4184e390b9 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:40.305239 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c444cc33-7b34-4778-a385-084142446405 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:40.310852 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f4317-0704-4ef6-ba86-7bd57d6b6cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:40.311654 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f4317-0704-4ef6-ba86-7bd57d6b6cec admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:40.312007 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f4317-0704-4ef6-ba86-7bd57d6b6cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:40.312498 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 808/1596] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:30:40.312653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f4317-0704-4ef6-ba86-7bd57d6b6cec admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:40.313178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f4317-0704-4ef6-ba86-7bd57d6b6cec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:40.313345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398872c-333e-4a2c-94c4-6f4184e390b9 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:40.314940 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f4317-0704-4ef6-ba86-7bd57d6b6cec admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:40.323124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398872c-333e-4a2c-94c4-6f4184e390b9 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:40.328074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 789/1597] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:30:40.328344 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f920a5-3235-4733-99d6-32b146276788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:40.329135 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f920a5-3235-4733-99d6-32b146276788 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:40.329611 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f4317-0704-4ef6-ba86-7bd57d6b6cec admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:40.330079 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f920a5-3235-4733-99d6-32b146276788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:40.330541 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f920a5-3235-4733-99d6-32b146276788 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:40.331369 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f920a5-3235-4733-99d6-32b146276788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:40.333258 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f920a5-3235-4733-99d6-32b146276788 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:40.341052 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f4317-0704-4ef6-ba86-7bd57d6b6cec admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:40.346841 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb8a83c-4fa4-4aca-9c6b-25b2760803c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:40.347008 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 809/1598] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:30:40.347982 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8a83c-4fa4-4aca-9c6b-25b2760803c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:40.348250 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2edf3e2-4f28-410e-818f-43cc1318af08 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:40.348536 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2edf3e2-4f28-410e-818f-43cc1318af08 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:40.348723 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8a83c-4fa4-4aca-9c6b-25b2760803c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:40.348927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2edf3e2-4f28-410e-818f-43cc1318af08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:40.349461 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8a83c-4fa4-4aca-9c6b-25b2760803c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:40.349982 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb8a83c-4fa4-4aca-9c6b-25b2760803c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:40.351709 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb8a83c-4fa4-4aca-9c6b-25b2760803c4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:40.363903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 790/1599] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:30:40.364747 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb8a83c-4fa4-4aca-9c6b-25b2760803c4 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:40.377256 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb8a83c-4fa4-4aca-9c6b-25b2760803c4 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:40.383029 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b2edf3e2-4f28-410e-818f-43cc1318af08 None None] MFA Rules not processed for user `cc722bd6c1ed489c8be32e20b599249c`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:40.384000 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d695b8-e625-4772-9d46-03a0702e9af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:40.384440 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b2edf3e2-4f28-410e-818f-43cc1318af08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:40.384742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d695b8-e625-4772-9d46-03a0702e9af8 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:40.385113 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d695b8-e625-4772-9d46-03a0702e9af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:40.385456 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d695b8-e625-4772-9d46-03a0702e9af8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:40.386180 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d695b8-e625-4772-9d46-03a0702e9af8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:40.388135 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d695b8-e625-4772-9d46-03a0702e9af8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:40.402403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 791/1600] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:30:40.403296 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d695b8-e625-4772-9d46-03a0702e9af8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:40.414102 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d695b8-e625-4772-9d46-03a0702e9af8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:40.419018 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbced7-c6fb-4d1e-b044-74f99f699a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:40.419615 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbced7-c6fb-4d1e-b044-74f99f699a17 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:40.419896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbced7-c6fb-4d1e-b044-74f99f699a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:40.420170 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbced7-c6fb-4d1e-b044-74f99f699a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:40.421441 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fbced7-c6fb-4d1e-b044-74f99f699a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:40.423078 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fbced7-c6fb-4d1e-b044-74f99f699a17 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:40.435558 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 792/1601] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:30:40.436265 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbced7-c6fb-4d1e-b044-74f99f699a17 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:40.449679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbced7-c6fb-4d1e-b044-74f99f699a17 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:40.455224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520ce01-710c-49c5-93ca-b01d0bb1646e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:40.455882 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520ce01-710c-49c5-93ca-b01d0bb1646e admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:40.456162 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520ce01-710c-49c5-93ca-b01d0bb1646e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:40.456760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520ce01-710c-49c5-93ca-b01d0bb1646e admin admin] PATH_INFO: `/v3/projects/4daf154c084546dfb637a68b0f708c6b/users/e16cc2e4e07b452889cc2d3346b32f3e/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:40.466254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 810/1602] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:40 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:40.466254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2edf3e2-4f28-410e-818f-43cc1318af08 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:40.473632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7520ce01-710c-49c5-93ca-b01d0bb1646e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4daf154c084546dfb637a68b0f708c6b, user_id=e16cc2e4e07b452889cc2d3346b32f3e, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:40.477220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7520ce01-710c-49c5-93ca-b01d0bb1646e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:40.479236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2edf3e2-4f28-410e-818f-43cc1318af08 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:40.487899 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c4563d-5fab-4309-a1c6-4f2c78c579c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:40.488727 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4563d-5fab-4309-a1c6-4f2c78c579c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:40.489088 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4563d-5fab-4309-a1c6-4f2c78c579c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:40.489479 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4563d-5fab-4309-a1c6-4f2c78c579c6 admin admin] PATH_INFO: `/v3/projects/b62e057f688142fba6a242a424bd4522/users/a0868f941e5e4d2184ab95316f7075d9/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:40.493480 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 793/1603] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:30:40 2022] PUT /identity/v3/projects/4daf154c084546dfb637a68b0f708c6b/users/e16cc2e4e07b452889cc2d3346b32f3e/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:30:40.494410 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520ce01-710c-49c5-93ca-b01d0bb1646e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:40.517421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c4563d-5fab-4309-a1c6-4f2c78c579c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b62e057f688142fba6a242a424bd4522, user_id=a0868f941e5e4d2184ab95316f7075d9, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:40.520274 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c4563d-5fab-4309-a1c6-4f2c78c579c6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:40.550118 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 811/1604] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:30:40 2022] PUT /identity/v3/projects/b62e057f688142fba6a242a424bd4522/users/a0868f941e5e4d2184ab95316f7075d9/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:30:40.551127 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c4563d-5fab-4309-a1c6-4f2c78c579c6 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:40.565666 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520ce01-710c-49c5-93ca-b01d0bb1646e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:40.579890 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:40.626259 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c4563d-5fab-4309-a1c6-4f2c78c579c6 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:40.631358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043aee6a-1c1b-4c2b-adc2-9ea9bab131a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:40.632023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043aee6a-1c1b-4c2b-adc2-9ea9bab131a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:40.632306 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043aee6a-1c1b-4c2b-adc2-9ea9bab131a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:40.632589 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043aee6a-1c1b-4c2b-adc2-9ea9bab131a0 admin admin] PATH_INFO: `/v3/projects/19bb3b3c67024a7bbe30ba1b59e7f50a/users/f69ac454b5a64e6d83188ad2b8e107d4/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:40.638276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4f664-7916-4ded-8e63-ae398da57898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:40.639790 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4f664-7916-4ded-8e63-ae398da57898 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:40.640249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4f664-7916-4ded-8e63-ae398da57898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:40.640531 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4f664-7916-4ded-8e63-ae398da57898 admin admin] PATH_INFO: `/v3/projects/7d5bb8b858e545b2bcede8b26a2188a3/users/024adeb8a8e14d678ae14174d9555152/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:40.645444 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:40.651488 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043aee6a-1c1b-4c2b-adc2-9ea9bab131a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19bb3b3c67024a7bbe30ba1b59e7f50a, user_id=f69ac454b5a64e6d83188ad2b8e107d4, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:40.654634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043aee6a-1c1b-4c2b-adc2-9ea9bab131a0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:40.659655 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4f664-7916-4ded-8e63-ae398da57898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d5bb8b858e545b2bcede8b26a2188a3, user_id=024adeb8a8e14d678ae14174d9555152, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:40.663004 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4f664-7916-4ded-8e63-ae398da57898 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:40.673590 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 812/1605] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:30:40 2022] PUT /identity/v3/projects/19bb3b3c67024a7bbe30ba1b59e7f50a/users/f69ac454b5a64e6d83188ad2b8e107d4/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:30:40.674718 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043aee6a-1c1b-4c2b-adc2-9ea9bab131a0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:40.685083 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 794/1606] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:30:40 2022] PUT /identity/v3/projects/7d5bb8b858e545b2bcede8b26a2188a3/users/024adeb8a8e14d678ae14174d9555152/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:30:40.687034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4f664-7916-4ded-8e63-ae398da57898 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:40.744553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043aee6a-1c1b-4c2b-adc2-9ea9bab131a0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:40.749798 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06781fd5-cd5e-49c6-948c-8bcdafff5db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:40.750448 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06781fd5-cd5e-49c6-948c-8bcdafff5db0 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:40.750728 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06781fd5-cd5e-49c6-948c-8bcdafff5db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:40.751002 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06781fd5-cd5e-49c6-948c-8bcdafff5db0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:40.751661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06781fd5-cd5e-49c6-948c-8bcdafff5db0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:40.751776 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4f664-7916-4ded-8e63-ae398da57898 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:40.753910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06781fd5-cd5e-49c6-948c-8bcdafff5db0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:40.756679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035ca79-8431-412d-a2d1-78aa85262411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:40.757364 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035ca79-8431-412d-a2d1-78aa85262411 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:40.757721 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035ca79-8431-412d-a2d1-78aa85262411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:40.758088 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035ca79-8431-412d-a2d1-78aa85262411 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:40.758778 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8035ca79-8431-412d-a2d1-78aa85262411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:40.760362 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8035ca79-8431-412d-a2d1-78aa85262411 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:40.767001 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 813/1607] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:30:40.767001 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06781fd5-cd5e-49c6-948c-8bcdafff5db0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:40.774060 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 795/1608] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:30:40.774944 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035ca79-8431-412d-a2d1-78aa85262411 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:40.778425 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06781fd5-cd5e-49c6-948c-8bcdafff5db0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:40.784819 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc37d5f-1873-434c-b03e-768df7861912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:40.785481 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc37d5f-1873-434c-b03e-768df7861912 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:40.785758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc37d5f-1873-434c-b03e-768df7861912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:40.786071 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc37d5f-1873-434c-b03e-768df7861912 admin admin] PATH_INFO: `/v3/projects/0b61e46f94b44e6d963e27ccae5dd165/users/4ecfc51400414558b8389a9c35b2fa1c/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:40.803064 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc37d5f-1873-434c-b03e-768df7861912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b61e46f94b44e6d963e27ccae5dd165, user_id=4ecfc51400414558b8389a9c35b2fa1c, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:40.806054 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc37d5f-1873-434c-b03e-768df7861912 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:40.827684 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 814/1609] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:30:40 2022] PUT /identity/v3/projects/0b61e46f94b44e6d963e27ccae5dd165/users/4ecfc51400414558b8389a9c35b2fa1c/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:30:40.829181 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd747b-2b91-487f-814e-aecffaa56321 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:40.829460 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd747b-2b91-487f-814e-aecffaa56321 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:40.829732 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd747b-2b91-487f-814e-aecffaa56321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:40.851788 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035ca79-8431-412d-a2d1-78aa85262411 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:40.859959 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-84fd747b-2b91-487f-814e-aecffaa56321 None None] MFA Rules not processed for user `e16cc2e4e07b452889cc2d3346b32f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:40.861165 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-84fd747b-2b91-487f-814e-aecffaa56321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:40.911355 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07d0ee-3332-4080-909a-9fbed3238800 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:40.912479 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07d0ee-3332-4080-909a-9fbed3238800 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:40.912761 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07d0ee-3332-4080-909a-9fbed3238800 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:40.913035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07d0ee-3332-4080-909a-9fbed3238800 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:40.919007 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:40.940155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 815/1610] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:40.945411 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c801564-98e4-4d85-a9dd-82535c429015 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:40.945411 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c801564-98e4-4d85-a9dd-82535c429015 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:40.945411 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c801564-98e4-4d85-a9dd-82535c429015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:40.984753 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-1c801564-98e4-4d85-a9dd-82535c429015 None None] MFA Rules not processed for user `a0868f941e5e4d2184ab95316f7075d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:40.986343 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1c801564-98e4-4d85-a9dd-82535c429015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:40.992672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07d0ee-3332-4080-909a-9fbed3238800 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:40.995772 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07d0ee-3332-4080-909a-9fbed3238800 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:41.007594 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 796/1611] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:40 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:41.008851 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db8bd3-6e1b-4457-8247-ca3b96e56830 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:41.009136 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db8bd3-6e1b-4457-8247-ca3b96e56830 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:41.009410 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db8bd3-6e1b-4457-8247-ca3b96e56830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:41.031699 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-01db8bd3-6e1b-4457-8247-ca3b96e56830 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:41.033066 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-01db8bd3-6e1b-4457-8247-ca3b96e56830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:41.041630 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:41.084384 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 816/1612] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:41.086429 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081f9f0-7634-46c6-8d2f-4d4a65331641 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:41.086716 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081f9f0-7634-46c6-8d2f-4d4a65331641 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:41.086992 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081f9f0-7634-46c6-8d2f-4d4a65331641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:41.109988 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e081f9f0-7634-46c6-8d2f-4d4a65331641 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:41.111981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e081f9f0-7634-46c6-8d2f-4d4a65331641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:41.129044 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 797/1613] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:41.129884 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db8bd3-6e1b-4457-8247-ca3b96e56830 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:41.148212 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db8bd3-6e1b-4457-8247-ca3b96e56830 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:41.154834 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a3441-b923-4805-9758-c7786f38b113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:41.155793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a3441-b923-4805-9758-c7786f38b113 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:41.156281 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a3441-b923-4805-9758-c7786f38b113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:41.156892 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a3441-b923-4805-9758-c7786f38b113 admin admin] PATH_INFO: `/v3/projects/27cd65c8d8044dddaf7a761697d216b4/users/0d4f8f3a6d42456d8e5cfbe20e54687a/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:41.173833 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a3441-b923-4805-9758-c7786f38b113 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27cd65c8d8044dddaf7a761697d216b4, user_id=0d4f8f3a6d42456d8e5cfbe20e54687a, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:41.176586 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a3441-b923-4805-9758-c7786f38b113 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:41.192982 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 798/1614] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:30:41 2022] PUT /identity/v3/projects/27cd65c8d8044dddaf7a761697d216b4/users/0d4f8f3a6d42456d8e5cfbe20e54687a/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:30:41.194160 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a3441-b923-4805-9758-c7786f38b113 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:41.197071 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 817/1615] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:41.198467 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748c97b-0c42-4e3f-9d2f-4fc412e3be14 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:41.198753 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748c97b-0c42-4e3f-9d2f-4fc412e3be14 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:41.199022 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748c97b-0c42-4e3f-9d2f-4fc412e3be14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:41.220539 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d748c97b-0c42-4e3f-9d2f-4fc412e3be14 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:41.221746 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d748c97b-0c42-4e3f-9d2f-4fc412e3be14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:41.278056 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a3441-b923-4805-9758-c7786f38b113 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:41.284898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d8321-18e3-48d8-a10b-f8a950eb8bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:41.284898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d8321-18e3-48d8-a10b-f8a950eb8bd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:41.284898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d8321-18e3-48d8-a10b-f8a950eb8bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:41.284898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d8321-18e3-48d8-a10b-f8a950eb8bd1 admin admin] PATH_INFO: `/v3/projects/10f3ba5ae75246ec9237a7e2305ae32a/users/a328e946d6d4441ca7ecbb3a20d2a611/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:41.291322 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 818/1616] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:41.296136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae70db4-2d9f-4765-a3ec-8a2757000670 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:41.296449 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae70db4-2d9f-4765-a3ec-8a2757000670 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:41.296724 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae70db4-2d9f-4765-a3ec-8a2757000670 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:41.298240 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 819/1617] 10.222.0.58 () {54 vars in 962 bytes} [Thu Sep 1 16:30:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:30:41.303168 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae70db4-2d9f-4765-a3ec-8a2757000670 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:41.303940 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d8321-18e3-48d8-a10b-f8a950eb8bd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10f3ba5ae75246ec9237a7e2305ae32a, user_id=a328e946d6d4441ca7ecbb3a20d2a611, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:41.306702 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d8321-18e3-48d8-a10b-f8a950eb8bd1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:41.317452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae70db4-2d9f-4765-a3ec-8a2757000670 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:41.326949 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 799/1618] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:30:41 2022] PUT /identity/v3/projects/10f3ba5ae75246ec9237a7e2305ae32a/users/a328e946d6d4441ca7ecbb3a20d2a611/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:30:41.328018 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d8321-18e3-48d8-a10b-f8a950eb8bd1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:41.335882 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:41.387949 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80298ace-bf6d-44f8-a547-bd86a72dc28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:41.388659 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80298ace-bf6d-44f8-a547-bd86a72dc28d admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:41.389011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80298ace-bf6d-44f8-a547-bd86a72dc28d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:41.389392 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80298ace-bf6d-44f8-a547-bd86a72dc28d admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:41.390103 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80298ace-bf6d-44f8-a547-bd86a72dc28d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:41.392370 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80298ace-bf6d-44f8-a547-bd86a72dc28d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:41.397271 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d8321-18e3-48d8-a10b-f8a950eb8bd1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:41.401994 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526ab09-375b-4b2d-958a-aca050ed8cd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:41.402679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526ab09-375b-4b2d-958a-aca050ed8cd1 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:41.402934 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526ab09-375b-4b2d-958a-aca050ed8cd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:41.403203 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526ab09-375b-4b2d-958a-aca050ed8cd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:41.423256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 820/1619] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:30:41 2022] POST /identity/v3/projects => generated 363 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:30:41.424150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80298ace-bf6d-44f8-a547-bd86a72dc28d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:41.437316 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:41.468962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526ab09-375b-4b2d-958a-aca050ed8cd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:41.470489 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526ab09-375b-4b2d-958a-aca050ed8cd1 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:41.480648 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 800/1620] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:41 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:41.481883 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c9f93-8164-41ab-9542-813e568c224d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:41.482167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c9f93-8164-41ab-9542-813e568c224d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:41.482467 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c9f93-8164-41ab-9542-813e568c224d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:41.495989 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80298ace-bf6d-44f8-a547-bd86a72dc28d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:41.501398 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8913603-1c3b-4190-9614-df9d06fe77f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:41.502041 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8913603-1c3b-4190-9614-df9d06fe77f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:41.502405 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8913603-1c3b-4190-9614-df9d06fe77f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:41.502757 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8913603-1c3b-4190-9614-df9d06fe77f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:41.504517 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-455c9f93-8164-41ab-9542-813e568c224d None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:41.505837 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-455c9f93-8164-41ab-9542-813e568c224d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:41.580385 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 801/1621] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:41.581551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c9f93-8164-41ab-9542-813e568c224d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:41.593090 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c9f93-8164-41ab-9542-813e568c224d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:41.596981 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47fcc9-2780-4872-85a1-9af88e425a79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:41.597710 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47fcc9-2780-4872-85a1-9af88e425a79 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:41.598121 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47fcc9-2780-4872-85a1-9af88e425a79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:41.598579 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47fcc9-2780-4872-85a1-9af88e425a79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:41.600235 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8913603-1c3b-4190-9614-df9d06fe77f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:41.602071 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8913603-1c3b-4190-9614-df9d06fe77f2 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:41.611883 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47fcc9-2780-4872-85a1-9af88e425a79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:41.614948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47fcc9-2780-4872-85a1-9af88e425a79 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:41.617341 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 821/1622] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:41.618522 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8913603-1c3b-4190-9614-df9d06fe77f2 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:41.627005 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 802/1623] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:41.628321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0afa7-11a0-4671-8db7-070d68820d3b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:41.628661 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0afa7-11a0-4671-8db7-070d68820d3b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:41.629311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0afa7-11a0-4671-8db7-070d68820d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:41.633766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8913603-1c3b-4190-9614-df9d06fe77f2 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:41.641279 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472b5a4-fe39-460e-b39d-052d8add46a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:41.642163 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472b5a4-fe39-460e-b39d-052d8add46a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:41.642574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472b5a4-fe39-460e-b39d-052d8add46a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:41.643572 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472b5a4-fe39-460e-b39d-052d8add46a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:41.654360 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0472b5a4-fe39-460e-b39d-052d8add46a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:41.659655 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0472b5a4-fe39-460e-b39d-052d8add46a6 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:41.677242 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 822/1624] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:41.679095 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f5fae-9788-4d7b-adf2-0ea43151b22b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:41.679483 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f5fae-9788-4d7b-adf2-0ea43151b22b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:41.679924 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f5fae-9788-4d7b-adf2-0ea43151b22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:41.705510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-74c0afa7-11a0-4671-8db7-070d68820d3b None None] MFA Rules not processed for user `1f0c99fdf8bd4331ad7e641ee38554c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:41.705750 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ad8f5fae-9788-4d7b-adf2-0ea43151b22b None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:41.708537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-74c0afa7-11a0-4671-8db7-070d68820d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:41.708687 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ad8f5fae-9788-4d7b-adf2-0ea43151b22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:41.726636 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 823/1625] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:41.727617 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f5fae-9788-4d7b-adf2-0ea43151b22b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:41.741535 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f5fae-9788-4d7b-adf2-0ea43151b22b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:41.751906 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f6b79-1023-46b6-a9d5-35e3bdd27885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:41.753300 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f6b79-1023-46b6-a9d5-35e3bdd27885 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:41.755893 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f6b79-1023-46b6-a9d5-35e3bdd27885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:41.755893 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f6b79-1023-46b6-a9d5-35e3bdd27885 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:41.755893 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f6b79-1023-46b6-a9d5-35e3bdd27885 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:41.758683 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:41.759728 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f6b79-1023-46b6-a9d5-35e3bdd27885 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:41.792305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 824/1626] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:30:41 2022] POST /identity/v3/users => generated 288 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:30:41.792835 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f6b79-1023-46b6-a9d5-35e3bdd27885 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:41.807509 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f6b79-1023-46b6-a9d5-35e3bdd27885 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:41.813406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0a50c-15a3-43ed-b308-bed727b47d0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:41.814019 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0a50c-15a3-43ed-b308-bed727b47d0c service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:41.814335 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0a50c-15a3-43ed-b308-bed727b47d0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:41.814627 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0a50c-15a3-43ed-b308-bed727b47d0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:41.831776 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 803/1627] 10.222.0.58 () {58 vars in 1059 bytes} [Thu Sep 1 16:30:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:41.832908 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a0a50c-15a3-43ed-b308-bed727b47d0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:41.833159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0afa7-11a0-4671-8db7-070d68820d3b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:41.834765 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a0a50c-15a3-43ed-b308-bed727b47d0c service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:41.835628 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-74c0afa7-11a0-4671-8db7-070d68820d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:41.857534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 825/1628] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:41.859265 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0a50c-15a3-43ed-b308-bed727b47d0c service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:41.862240 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0afa7-11a0-4671-8db7-070d68820d3b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:41.871319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0a50c-15a3-43ed-b308-bed727b47d0c service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:41.876272 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee11315-5afe-49f4-898f-3067da3592c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:41.876434 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd6d94-3499-43f5-b771-9ce3e65fd240 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:41.876888 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee11315-5afe-49f4-898f-3067da3592c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:41.877169 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee11315-5afe-49f4-898f-3067da3592c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:41.877386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd6d94-3499-43f5-b771-9ce3e65fd240 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:41.877498 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee11315-5afe-49f4-898f-3067da3592c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:41.877933 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd6d94-3499-43f5-b771-9ce3e65fd240 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:41.878278 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd6d94-3499-43f5-b771-9ce3e65fd240 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:41.878540 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee11315-5afe-49f4-898f-3067da3592c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:41.881487 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee11315-5afe-49f4-898f-3067da3592c9 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:41.901125 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 826/1629] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:30:41.902384 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee11315-5afe-49f4-898f-3067da3592c9 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:41.914000 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd6d94-3499-43f5-b771-9ce3e65fd240 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:41.916654 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd6d94-3499-43f5-b771-9ce3e65fd240 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:41.923997 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee11315-5afe-49f4-898f-3067da3592c9 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:41.934803 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf16f80e-9a36-4011-8ca7-b6dfc0a5a287 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:41.935869 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16f80e-9a36-4011-8ca7-b6dfc0a5a287 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:41.936255 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16f80e-9a36-4011-8ca7-b6dfc0a5a287 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:41.936733 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16f80e-9a36-4011-8ca7-b6dfc0a5a287 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:41.943424 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 804/1630] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:41.944494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd6d94-3499-43f5-b771-9ce3e65fd240 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:41.965340 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd6d94-3499-43f5-b771-9ce3e65fd240 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:41.981127 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ecc046-b427-4e63-9bfe-bcca9cb5e718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:41.982480 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecc046-b427-4e63-9bfe-bcca9cb5e718 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:41.988844 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecc046-b427-4e63-9bfe-bcca9cb5e718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:41.988844 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecc046-b427-4e63-9bfe-bcca9cb5e718 admin admin] PATH_INFO: `/v3/projects/1a7a7f7b15b84a47a9cac1a2520e442f/users/41611648338f4b2192bd373a49e8431b/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:42.037701 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ecc046-b427-4e63-9bfe-bcca9cb5e718 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a7a7f7b15b84a47a9cac1a2520e442f, user_id=41611648338f4b2192bd373a49e8431b, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:42.057695 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ecc046-b427-4e63-9bfe-bcca9cb5e718 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:42.105319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 805/1631] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:30:41 2022] PUT /identity/v3/projects/1a7a7f7b15b84a47a9cac1a2520e442f/users/41611648338f4b2192bd373a49e8431b/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:30:42.106774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ecc046-b427-4e63-9bfe-bcca9cb5e718 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:42.113727 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:42.126141 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf16f80e-9a36-4011-8ca7-b6dfc0a5a287 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:42.127816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf16f80e-9a36-4011-8ca7-b6dfc0a5a287 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:42.162787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 827/1632] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:42.321809 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ecc046-b427-4e63-9bfe-bcca9cb5e718 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:42.344479 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de8c17-79cc-49e4-a70a-6bbc010aede7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:42.344479 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de8c17-79cc-49e4-a70a-6bbc010aede7 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:42.345117 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de8c17-79cc-49e4-a70a-6bbc010aede7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:42.345498 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de8c17-79cc-49e4-a70a-6bbc010aede7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:42.346278 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de8c17-79cc-49e4-a70a-6bbc010aede7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:42.352988 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de8c17-79cc-49e4-a70a-6bbc010aede7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:42.378747 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 806/1633] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:30:42.382715 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf16f80e-9a36-4011-8ca7-b6dfc0a5a287 service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:42.409399 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf16f80e-9a36-4011-8ca7-b6dfc0a5a287 service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:42.431136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46779326-b7f5-47b4-b709-91ae6f137388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:42.440433 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46779326-b7f5-47b4-b709-91ae6f137388 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:42.440572 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46779326-b7f5-47b4-b709-91ae6f137388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:42.444127 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46779326-b7f5-47b4-b709-91ae6f137388 admin admin] PATH_INFO: `/v3/projects/1a7a7f7b15b84a47a9cac1a2520e442f/users/41611648338f4b2192bd373a49e8431b/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:42.457225 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46779326-b7f5-47b4-b709-91ae6f137388 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a7a7f7b15b84a47a9cac1a2520e442f, user_id=41611648338f4b2192bd373a49e8431b, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:42.470241 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46779326-b7f5-47b4-b709-91ae6f137388 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:42.537254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 828/1634] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:30:42 2022] PUT /identity/v3/projects/1a7a7f7b15b84a47a9cac1a2520e442f/users/41611648338f4b2192bd373a49e8431b/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:30:42.548126 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e928de0-08c4-489b-8d49-b17a5c1d46ed None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:42.548126 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e928de0-08c4-489b-8d49-b17a5c1d46ed None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:42.548126 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e928de0-08c4-489b-8d49-b17a5c1d46ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:42.611204 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2e928de0-08c4-489b-8d49-b17a5c1d46ed None None] MFA Rules not processed for user `41611648338f4b2192bd373a49e8431b`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:42.613714 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2e928de0-08c4-489b-8d49-b17a5c1d46ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:42.785699 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 807/1635] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:42 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:42.798553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46779326-b7f5-47b4-b709-91ae6f137388 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:42.853200 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d335ca0-55f1-4def-aa7d-f15e37a96304 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:42.854940 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d335ca0-55f1-4def-aa7d-f15e37a96304 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:42.855592 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d335ca0-55f1-4def-aa7d-f15e37a96304 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:42.857726 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 808/1636] 10.222.0.58 () {54 vars in 912 bytes} [Thu Sep 1 16:30:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:30:42.865658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d4b68-02dc-4ba6-96b2-ea93d6effe30 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:42.872161 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d4b68-02dc-4ba6-96b2-ea93d6effe30 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:42.872977 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d4b68-02dc-4ba6-96b2-ea93d6effe30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:42.901756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:42.919082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46779326-b7f5-47b4-b709-91ae6f137388 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:42.921781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-495d4b68-02dc-4ba6-96b2-ea93d6effe30 None None] MFA Rules not processed for user `1f0c99fdf8bd4331ad7e641ee38554c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:42.923962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-495d4b68-02dc-4ba6-96b2-ea93d6effe30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:42.932009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be85815-2cd4-4098-9d4f-f7132c26f195 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:42.934855 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be85815-2cd4-4098-9d4f-f7132c26f195 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:42.935395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be85815-2cd4-4098-9d4f-f7132c26f195 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:42.938650 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be85815-2cd4-4098-9d4f-f7132c26f195 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:42.963833 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be85815-2cd4-4098-9d4f-f7132c26f195 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:42.967406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be85815-2cd4-4098-9d4f-f7132c26f195 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:42.997518 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 829/1637] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:42 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:30:43.088678 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 809/1638] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Sep 1 16:30:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:30:43.102819 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be85815-2cd4-4098-9d4f-f7132c26f195 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:43.114059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be85815-2cd4-4098-9d4f-f7132c26f195 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:43.122858 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef5a252-3e12-4750-9446-884064ce3e07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:43.125267 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef5a252-3e12-4750-9446-884064ce3e07 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:43.126785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef5a252-3e12-4750-9446-884064ce3e07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:43.126785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef5a252-3e12-4750-9446-884064ce3e07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:43.147410 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef5a252-3e12-4750-9446-884064ce3e07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:43.149068 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef5a252-3e12-4750-9446-884064ce3e07 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:43.166449 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 830/1639] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:43.203875 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d4b68-02dc-4ba6-96b2-ea93d6effe30 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:43.226351 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d4b68-02dc-4ba6-96b2-ea93d6effe30 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:43.231411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba505ac-abd6-44d9-b32a-c3875ca935bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:43.232945 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba505ac-abd6-44d9-b32a-c3875ca935bd service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:43.235659 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba505ac-abd6-44d9-b32a-c3875ca935bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:43.236521 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba505ac-abd6-44d9-b32a-c3875ca935bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:43.345793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba505ac-abd6-44d9-b32a-c3875ca935bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:43.348323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba505ac-abd6-44d9-b32a-c3875ca935bd service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:43.418238 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 810/1640] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:44.468191 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c62b4-9b62-477a-a52f-cf1a9387a728 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:44.469607 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c62b4-9b62-477a-a52f-cf1a9387a728 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:44.470245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c62b4-9b62-477a-a52f-cf1a9387a728 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:44.478814 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 831/1641] 10.222.0.58 () {54 vars in 962 bytes} [Thu Sep 1 16:30:44 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:30:44.493847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6f552-ae70-4834-a436-e90a9f651ad6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:44.494529 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6f552-ae70-4834-a436-e90a9f651ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:44.494529 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6f552-ae70-4834-a436-e90a9f651ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:44.536386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-97a6f552-ae70-4834-a436-e90a9f651ad6 None None] MFA Rules not processed for user `1f0c99fdf8bd4331ad7e641ee38554c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:44.541189 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-97a6f552-ae70-4834-a436-e90a9f651ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:44.566089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 811/1642] 10.222.0.58 () {58 vars in 1059 bytes} [Thu Sep 1 16:30:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:30:44.572090 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00549e-7655-43e9-b354-52d97deae5d5 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:44.572090 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00549e-7655-43e9-b354-52d97deae5d5 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:44.572413 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00549e-7655-43e9-b354-52d97deae5d5 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:44.573904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 832/1643] 10.222.0.58 () {54 vars in 912 bytes} [Thu Sep 1 16:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:30:44.581292 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c65de-fddb-4b26-a6a4-87900660f210 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:44.581571 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c65de-fddb-4b26-a6a4-87900660f210 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:44.581852 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c65de-fddb-4b26-a6a4-87900660f210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:44.624196 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4f2c65de-fddb-4b26-a6a4-87900660f210 None None] MFA Rules not processed for user `1f0c99fdf8bd4331ad7e641ee38554c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:44.628040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4f2c65de-fddb-4b26-a6a4-87900660f210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:44.647415 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 812/1644] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Sep 1 16:30:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:44.664988 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00549e-7655-43e9-b354-52d97deae5d5 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:44.686111 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00549e-7655-43e9-b354-52d97deae5d5 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:44.704064 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb56e9b-a0ab-4b97-b33b-8432856f955b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:44.704800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb56e9b-a0ab-4b97-b33b-8432856f955b service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:44.705046 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb56e9b-a0ab-4b97-b33b-8432856f955b service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:44.705316 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb56e9b-a0ab-4b97-b33b-8432856f955b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:44.731138 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb56e9b-a0ab-4b97-b33b-8432856f955b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:44.733519 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb56e9b-a0ab-4b97-b33b-8432856f955b service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:44.761355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 833/1645] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:46.715508 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c65de-fddb-4b26-a6a4-87900660f210 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:46.930478 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c65de-fddb-4b26-a6a4-87900660f210 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:46.936182 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9add0-057e-4a05-a8db-5f02da25871d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:46.936912 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9add0-057e-4a05-a8db-5f02da25871d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:46.937162 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9add0-057e-4a05-a8db-5f02da25871d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:46.937472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9add0-057e-4a05-a8db-5f02da25871d admin admin] PATH_INFO: `/v3/users/e16cc2e4e07b452889cc2d3346b32f3e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:46.938691 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9add0-057e-4a05-a8db-5f02da25871d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e16cc2e4e07b452889cc2d3346b32f3e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:46.940035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-52e9add0-057e-4a05-a8db-5f02da25871d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:30:46.940035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Sep 01 16:30:46.940035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: Sep 01 16:30:46.942044 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9add0-057e-4a05-a8db-5f02da25871d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:47.029239 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:47.090767 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-52e9add0-057e-4a05-a8db-5f02da25871d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e16cc2e4e07b452889cc2d3346b32f3e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:47.117717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-52e9add0-057e-4a05-a8db-5f02da25871d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e16cc2e4e07b452889cc2d3346b32f3e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:47.120421 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-52e9add0-057e-4a05-a8db-5f02da25871d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e16cc2e4e07b452889cc2d3346b32f3e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:47.141892 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-52e9add0-057e-4a05-a8db-5f02da25871d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e16cc2e4e07b452889cc2d3346b32f3e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:47.181328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-52e9add0-057e-4a05-a8db-5f02da25871d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e16cc2e4e07b452889cc2d3346b32f3e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:47.196241 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 813/1646] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:30:46 2022] DELETE /identity/v3/users/e16cc2e4e07b452889cc2d3346b32f3e => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:30:47.204466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a8049-9905-4570-83ed-4d24a7944509 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:47.205181 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a8049-9905-4570-83ed-4d24a7944509 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:47.206703 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a8049-9905-4570-83ed-4d24a7944509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:47.259067 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-4d2a8049-9905-4570-83ed-4d24a7944509 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:47.261494 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4d2a8049-9905-4570-83ed-4d24a7944509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:47.483954 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 834/1647] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:47.686222 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9add0-057e-4a05-a8db-5f02da25871d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:47.688065 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-52e9add0-057e-4a05-a8db-5f02da25871d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:48.065567 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9add0-057e-4a05-a8db-5f02da25871d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:48.080440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc70fc7-4169-4f37-aa8c-3ce575e4d34c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:48.081085 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc70fc7-4169-4f37-aa8c-3ce575e4d34c service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:48.081373 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc70fc7-4169-4f37-aa8c-3ce575e4d34c service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:48.081654 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc70fc7-4169-4f37-aa8c-3ce575e4d34c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:48.140361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc70fc7-4169-4f37-aa8c-3ce575e4d34c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:48.141819 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc70fc7-4169-4f37-aa8c-3ce575e4d34c service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:48.173962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 814/1648] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:49.472601 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a8049-9905-4570-83ed-4d24a7944509 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:49.474316 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4d2a8049-9905-4570-83ed-4d24a7944509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:49.497355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a8049-9905-4570-83ed-4d24a7944509 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:49.513894 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82855308-619b-488a-8226-e82b9b5beb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:49.515430 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82855308-619b-488a-8226-e82b9b5beb48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:49.515729 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82855308-619b-488a-8226-e82b9b5beb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:49.516270 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82855308-619b-488a-8226-e82b9b5beb48 admin admin] PATH_INFO: `/v3/users/0afe9571b2304659b7be359fbda93d2c` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:49.517654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82855308-619b-488a-8226-e82b9b5beb48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0afe9571b2304659b7be359fbda93d2c)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:49.520068 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82855308-619b-488a-8226-e82b9b5beb48 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:49.675386 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-82855308-619b-488a-8226-e82b9b5beb48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0afe9571b2304659b7be359fbda93d2c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:49.690840 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:49.719285 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-82855308-619b-488a-8226-e82b9b5beb48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0afe9571b2304659b7be359fbda93d2c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:49.720998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-82855308-619b-488a-8226-e82b9b5beb48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0afe9571b2304659b7be359fbda93d2c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:49.724009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-82855308-619b-488a-8226-e82b9b5beb48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0afe9571b2304659b7be359fbda93d2c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:49.766670 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-82855308-619b-488a-8226-e82b9b5beb48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0afe9571b2304659b7be359fbda93d2c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:49.799399 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 835/1649] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:30:49 2022] DELETE /identity/v3/users/0afe9571b2304659b7be359fbda93d2c => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:30:49.803562 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2cc23-051b-4f6f-9c67-b8d0e8d100f4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:49.804202 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2cc23-051b-4f6f-9c67-b8d0e8d100f4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:49.804691 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2cc23-051b-4f6f-9c67-b8d0e8d100f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:49.849023 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3bd2cc23-051b-4f6f-9c67-b8d0e8d100f4 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:49.852985 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3bd2cc23-051b-4f6f-9c67-b8d0e8d100f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:50.040671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 815/1650] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:50.116609 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82855308-619b-488a-8226-e82b9b5beb48 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:50.118846 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-82855308-619b-488a-8226-e82b9b5beb48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:50.215782 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82855308-619b-488a-8226-e82b9b5beb48 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:50.220699 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee711252-be9f-4dc6-85a3-3be118b09c72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:50.221556 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee711252-be9f-4dc6-85a3-3be118b09c72 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:50.222819 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee711252-be9f-4dc6-85a3-3be118b09c72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:50.223564 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee711252-be9f-4dc6-85a3-3be118b09c72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:50.241431 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee711252-be9f-4dc6-85a3-3be118b09c72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:50.243284 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee711252-be9f-4dc6-85a3-3be118b09c72 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:50.260461 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 836/1651] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:50.493347 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd2cc23-051b-4f6f-9c67-b8d0e8d100f4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:50.494680 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3bd2cc23-051b-4f6f-9c67-b8d0e8d100f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:50.512626 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd2cc23-051b-4f6f-9c67-b8d0e8d100f4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:50.526533 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218038f-6929-4298-b1e5-65e67ce06f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:50.527450 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218038f-6929-4298-b1e5-65e67ce06f25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:50.527749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218038f-6929-4298-b1e5-65e67ce06f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:50.528032 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218038f-6929-4298-b1e5-65e67ce06f25 admin admin] PATH_INFO: `/v3/projects/4daf154c084546dfb637a68b0f708c6b` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:50.530905 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1218038f-6929-4298-b1e5-65e67ce06f25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4daf154c084546dfb637a68b0f708c6b)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:50.532873 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1218038f-6929-4298-b1e5-65e67ce06f25 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:50.656619 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:50.676696 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1218038f-6929-4298-b1e5-65e67ce06f25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4daf154c084546dfb637a68b0f708c6b', 'request_id': 'req-1218038f-6929-4298-b1e5-65e67ce06f25'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:50.693600 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1218038f-6929-4298-b1e5-65e67ce06f25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4daf154c084546dfb637a68b0f708c6b', 'request_id': 'req-1218038f-6929-4298-b1e5-65e67ce06f25'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:50.721054 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1218038f-6929-4298-b1e5-65e67ce06f25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4daf154c084546dfb637a68b0f708c6b', 'request_id': 'req-1218038f-6929-4298-b1e5-65e67ce06f25'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:50.738124 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1218038f-6929-4298-b1e5-65e67ce06f25 admin admin] The token cache is being invalidate because project 4daf154c084546dfb637a68b0f708c6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:30:50.740538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1218038f-6929-4298-b1e5-65e67ce06f25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:50.741679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1218038f-6929-4298-b1e5-65e67ce06f25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:50.754518 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 816/1652] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:30:50 2022] DELETE /identity/v3/projects/4daf154c084546dfb637a68b0f708c6b => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:30:50.759311 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc962a-5421-4c85-8267-094789d59029 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:50.760402 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc962a-5421-4c85-8267-094789d59029 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:50.760822 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc962a-5421-4c85-8267-094789d59029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:50.805526 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a3cc962a-5421-4c85-8267-094789d59029 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:50.808698 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a3cc962a-5421-4c85-8267-094789d59029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:50.987378 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 837/1653] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:50.995272 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a76b68-7220-4437-a3ef-3f1807fd4dc3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:50.996192 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a76b68-7220-4437-a3ef-3f1807fd4dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:50.996626 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a76b68-7220-4437-a3ef-3f1807fd4dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:51.042871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-21a76b68-7220-4437-a3ef-3f1807fd4dc3 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:51.044359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-21a76b68-7220-4437-a3ef-3f1807fd4dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:51.066740 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 817/1654] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:51.071237 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc962a-5421-4c85-8267-094789d59029 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:51.086365 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc962a-5421-4c85-8267-094789d59029 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:51.105678 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429bdb7e-8288-4e28-ae49-624afbabb844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:51.106324 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429bdb7e-8288-4e28-ae49-624afbabb844 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:51.106609 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429bdb7e-8288-4e28-ae49-624afbabb844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:51.107452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429bdb7e-8288-4e28-ae49-624afbabb844 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:51.108161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429bdb7e-8288-4e28-ae49-624afbabb844 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:30:51.108460 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429bdb7e-8288-4e28-ae49-624afbabb844 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:51.110121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429bdb7e-8288-4e28-ae49-624afbabb844 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:51.131865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 838/1655] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:30:51.136113 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a76b68-7220-4437-a3ef-3f1807fd4dc3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:51.146187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a76b68-7220-4437-a3ef-3f1807fd4dc3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:51.158035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a99fb-b3f2-449f-b5ef-f712a8292e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:51.158937 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a99fb-b3f2-449f-b5ef-f712a8292e24 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:51.159368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a99fb-b3f2-449f-b5ef-f712a8292e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:51.159772 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a99fb-b3f2-449f-b5ef-f712a8292e24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:51.160558 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a99fb-b3f2-449f-b5ef-f712a8292e24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:51.163117 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a99fb-b3f2-449f-b5ef-f712a8292e24 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:51.259304 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 818/1656] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:30:51 2022] POST /identity/v3/projects => generated 367 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:30:51.262901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429bdb7e-8288-4e28-ae49-624afbabb844 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:51.275056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429bdb7e-8288-4e28-ae49-624afbabb844 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:51.443779 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b4a64-5fa3-4fec-b97f-d62b17a79f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:51.444407 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b4a64-5fa3-4fec-b97f-d62b17a79f82 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:51.444694 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b4a64-5fa3-4fec-b97f-d62b17a79f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:51.444967 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b4a64-5fa3-4fec-b97f-d62b17a79f82 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:51.446353 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9b4a64-5fa3-4fec-b97f-d62b17a79f82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:51.456016 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9b4a64-5fa3-4fec-b97f-d62b17a79f82 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:51.470633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:51.520201 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 839/1657] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:30:51 2022] POST /identity/v3/users => generated 284 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:30:51.525380 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a99fb-b3f2-449f-b5ef-f712a8292e24 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:51.555757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a99fb-b3f2-449f-b5ef-f712a8292e24 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:51.572197 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db764eb8-5b52-4d52-8668-46eac2a753cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:51.573175 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db764eb8-5b52-4d52-8668-46eac2a753cf admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:51.573816 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db764eb8-5b52-4d52-8668-46eac2a753cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:51.575021 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db764eb8-5b52-4d52-8668-46eac2a753cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:51.580786 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db764eb8-5b52-4d52-8668-46eac2a753cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:51.584494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db764eb8-5b52-4d52-8668-46eac2a753cf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:51.621030 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 819/1658] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:30:51.624377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b4a64-5fa3-4fec-b97f-d62b17a79f82 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:51.643706 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b4a64-5fa3-4fec-b97f-d62b17a79f82 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:51.670770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5558bcb-21a6-48a7-93c5-4eb84140041d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:51.671708 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5558bcb-21a6-48a7-93c5-4eb84140041d admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:51.672095 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5558bcb-21a6-48a7-93c5-4eb84140041d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:51.672475 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5558bcb-21a6-48a7-93c5-4eb84140041d admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:51.673245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5558bcb-21a6-48a7-93c5-4eb84140041d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:51.677217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5558bcb-21a6-48a7-93c5-4eb84140041d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:51.727023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 840/1659] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:30:51.730067 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db764eb8-5b52-4d52-8668-46eac2a753cf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:51.762058 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db764eb8-5b52-4d52-8668-46eac2a753cf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:51.777469 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49faf2c-22dd-45ea-b7dd-7ec8802f90bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:51.779586 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49faf2c-22dd-45ea-b7dd-7ec8802f90bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:51.779586 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49faf2c-22dd-45ea-b7dd-7ec8802f90bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:51.779586 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49faf2c-22dd-45ea-b7dd-7ec8802f90bc admin admin] PATH_INFO: `/v3/projects/f0d017e537c048c284664397d7ffd260/users/8bf3787ee7ab4063b3edff49ab345369/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:51.823862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49faf2c-22dd-45ea-b7dd-7ec8802f90bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0d017e537c048c284664397d7ffd260, user_id=8bf3787ee7ab4063b3edff49ab345369, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:51.835469 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49faf2c-22dd-45ea-b7dd-7ec8802f90bc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:51.896513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a1dc2-2a18-4451-bc5b-ce053ea2371b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:51.896513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a1dc2-2a18-4451-bc5b-ce053ea2371b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:51.896513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a1dc2-2a18-4451-bc5b-ce053ea2371b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:51.897131 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 820/1660] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:30:51 2022] PUT /identity/v3/projects/f0d017e537c048c284664397d7ffd260/users/8bf3787ee7ab4063b3edff49ab345369/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:30:51.954230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-447a1dc2-2a18-4451-bc5b-ce053ea2371b None None] MFA Rules not processed for user `8bf3787ee7ab4063b3edff49ab345369`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:51.955371 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-447a1dc2-2a18-4451-bc5b-ce053ea2371b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:52.180465 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49faf2c-22dd-45ea-b7dd-7ec8802f90bc admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:52.183318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e49faf2c-22dd-45ea-b7dd-7ec8802f90bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:52.185221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 841/1661] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:52.299965 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49faf2c-22dd-45ea-b7dd-7ec8802f90bc admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:52.305949 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb92b00-f3e6-4ad4-9d5a-1831d00041b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:52.306796 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb92b00-f3e6-4ad4-9d5a-1831d00041b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:52.307086 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb92b00-f3e6-4ad4-9d5a-1831d00041b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:52.308034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb92b00-f3e6-4ad4-9d5a-1831d00041b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:52.318987 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb92b00-f3e6-4ad4-9d5a-1831d00041b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:52.320501 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb92b00-f3e6-4ad4-9d5a-1831d00041b0 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:52.358511 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 821/1662] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:53.375908 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a1dc2-2a18-4451-bc5b-ce053ea2371b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:53.377945 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-447a1dc2-2a18-4451-bc5b-ce053ea2371b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:53.567679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a1dc2-2a18-4451-bc5b-ce053ea2371b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:53.573401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4432d9-3c26-46a6-87c2-beef52c24a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:53.574065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4432d9-3c26-46a6-87c2-beef52c24a8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:53.574416 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4432d9-3c26-46a6-87c2-beef52c24a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:53.574694 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4432d9-3c26-46a6-87c2-beef52c24a8d admin admin] PATH_INFO: `/v3/projects/f3cd1605b6ee4e52ab0c2edab29acbe3` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:53.576869 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4432d9-3c26-46a6-87c2-beef52c24a8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3cd1605b6ee4e52ab0c2edab29acbe3)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:53.577910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: WARNING py.warnings [None req-3d4432d9-3c26-46a6-87c2-beef52c24a8d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:30:53.577910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: warnings.warn(msg) Sep 01 16:30:53.577910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: Sep 01 16:30:53.583395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4432d9-3c26-46a6-87c2-beef52c24a8d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:53.747090 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3d4432d9-3c26-46a6-87c2-beef52c24a8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3cd1605b6ee4e52ab0c2edab29acbe3', 'request_id': 'req-3d4432d9-3c26-46a6-87c2-beef52c24a8d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:53.775222 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3d4432d9-3c26-46a6-87c2-beef52c24a8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3cd1605b6ee4e52ab0c2edab29acbe3', 'request_id': 'req-3d4432d9-3c26-46a6-87c2-beef52c24a8d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:53.787065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3d4432d9-3c26-46a6-87c2-beef52c24a8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3cd1605b6ee4e52ab0c2edab29acbe3', 'request_id': 'req-3d4432d9-3c26-46a6-87c2-beef52c24a8d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:53.811504 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:53.816636 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3d4432d9-3c26-46a6-87c2-beef52c24a8d admin admin] The token cache is being invalidate because project f3cd1605b6ee4e52ab0c2edab29acbe3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:30:53.816939 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3d4432d9-3c26-46a6-87c2-beef52c24a8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:53.819387 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3d4432d9-3c26-46a6-87c2-beef52c24a8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:53.824675 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 842/1663] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:30:53 2022] DELETE /identity/v3/projects/f3cd1605b6ee4e52ab0c2edab29acbe3 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:30:53.825633 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb92b00-f3e6-4ad4-9d5a-1831d00041b0 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:53.844950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb92b00-f3e6-4ad4-9d5a-1831d00041b0 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:53.975152 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8bad1a-9d35-4d05-aefc-1686202be46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:53.978465 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8bad1a-9d35-4d05-aefc-1686202be46e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:53.982092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8bad1a-9d35-4d05-aefc-1686202be46e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:53.983241 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8bad1a-9d35-4d05-aefc-1686202be46e admin admin] PATH_INFO: `/v3/users/cc722bd6c1ed489c8be32e20b599249c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:53.986518 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8bad1a-9d35-4d05-aefc-1686202be46e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc722bd6c1ed489c8be32e20b599249c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:53.988681 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8bad1a-9d35-4d05-aefc-1686202be46e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:54.020698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:54.155707 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-dd8bad1a-9d35-4d05-aefc-1686202be46e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc722bd6c1ed489c8be32e20b599249c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:54.189370 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-dd8bad1a-9d35-4d05-aefc-1686202be46e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc722bd6c1ed489c8be32e20b599249c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:54.190724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-dd8bad1a-9d35-4d05-aefc-1686202be46e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc722bd6c1ed489c8be32e20b599249c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:54.219439 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-dd8bad1a-9d35-4d05-aefc-1686202be46e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc722bd6c1ed489c8be32e20b599249c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:54.269416 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-dd8bad1a-9d35-4d05-aefc-1686202be46e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc722bd6c1ed489c8be32e20b599249c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:54.279184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 822/1664] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:30:53 2022] DELETE /identity/v3/users/cc722bd6c1ed489c8be32e20b599249c => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:30:56.443477 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4432d9-3c26-46a6-87c2-beef52c24a8d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:56.445263 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3d4432d9-3c26-46a6-87c2-beef52c24a8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:56.471669 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4432d9-3c26-46a6-87c2-beef52c24a8d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:56.648113 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c3195-4002-4426-b0f5-470a9ac51748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:56.648888 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c3195-4002-4426-b0f5-470a9ac51748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:56.649239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c3195-4002-4426-b0f5-470a9ac51748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:56.650517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c3195-4002-4426-b0f5-470a9ac51748 admin admin] PATH_INFO: `/v3/projects/55b0149dfe5146ee955a0bb80784e064` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:56.655439 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c3195-4002-4426-b0f5-470a9ac51748 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55b0149dfe5146ee955a0bb80784e064)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:56.657491 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c3195-4002-4426-b0f5-470a9ac51748 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:56.813728 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3d1c3195-4002-4426-b0f5-470a9ac51748 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55b0149dfe5146ee955a0bb80784e064', 'request_id': 'req-3d1c3195-4002-4426-b0f5-470a9ac51748'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:56.829464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3d1c3195-4002-4426-b0f5-470a9ac51748 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55b0149dfe5146ee955a0bb80784e064', 'request_id': 'req-3d1c3195-4002-4426-b0f5-470a9ac51748'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:56.848853 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3d1c3195-4002-4426-b0f5-470a9ac51748 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55b0149dfe5146ee955a0bb80784e064', 'request_id': 'req-3d1c3195-4002-4426-b0f5-470a9ac51748'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:56.881293 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3d1c3195-4002-4426-b0f5-470a9ac51748 admin admin] The token cache is being invalidate because project 55b0149dfe5146ee955a0bb80784e064 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:30:56.881646 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3d1c3195-4002-4426-b0f5-470a9ac51748 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:56.882430 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3d1c3195-4002-4426-b0f5-470a9ac51748 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:56.885077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 843/1665] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:30:56 2022] DELETE /identity/v3/projects/55b0149dfe5146ee955a0bb80784e064 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:30:56.890350 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41d80e-febd-4fd4-a534-71788a8117c2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:56.891235 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41d80e-febd-4fd4-a534-71788a8117c2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:56.891681 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41d80e-febd-4fd4-a534-71788a8117c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:56.933639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ae41d80e-febd-4fd4-a534-71788a8117c2 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:56.944260 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ae41d80e-febd-4fd4-a534-71788a8117c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:57.128638 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 823/1666] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:57.139220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a971d3-56dd-4e6b-b38f-9f56d9d41e9a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:57.139220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a971d3-56dd-4e6b-b38f-9f56d9d41e9a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:57.139220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a971d3-56dd-4e6b-b38f-9f56d9d41e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:57.184128 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-39a971d3-56dd-4e6b-b38f-9f56d9d41e9a None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:57.185942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-39a971d3-56dd-4e6b-b38f-9f56d9d41e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:57.203512 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 844/1667] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:57.204689 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae41d80e-febd-4fd4-a534-71788a8117c2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:57.219206 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae41d80e-febd-4fd4-a534-71788a8117c2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:57.233238 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a28405-e7a5-4168-9d80-0ba1e35a6c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:57.233967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a28405-e7a5-4168-9d80-0ba1e35a6c0e admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:57.234275 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a28405-e7a5-4168-9d80-0ba1e35a6c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:57.234556 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a28405-e7a5-4168-9d80-0ba1e35a6c0e admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:57.235229 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a28405-e7a5-4168-9d80-0ba1e35a6c0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:30:57.235545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a28405-e7a5-4168-9d80-0ba1e35a6c0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:57.237182 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a28405-e7a5-4168-9d80-0ba1e35a6c0e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:57.275610 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 824/1668] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:30:57.277560 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a971d3-56dd-4e6b-b38f-9f56d9d41e9a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:57.289136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a971d3-56dd-4e6b-b38f-9f56d9d41e9a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:57.295386 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116013c-cca0-4391-b52e-e2807f0d6072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:57.296021 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116013c-cca0-4391-b52e-e2807f0d6072 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:57.296304 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116013c-cca0-4391-b52e-e2807f0d6072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:57.296582 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116013c-cca0-4391-b52e-e2807f0d6072 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:57.297286 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5116013c-cca0-4391-b52e-e2807f0d6072 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:57.299190 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5116013c-cca0-4391-b52e-e2807f0d6072 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:57.424557 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 845/1669] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:30:57 2022] POST /identity/v3/projects => generated 381 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:30:57.429106 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a28405-e7a5-4168-9d80-0ba1e35a6c0e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:57.496204 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a28405-e7a5-4168-9d80-0ba1e35a6c0e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:57.716855 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396c29e-dd79-4e52-9d6c-4dddfedb0777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:57.717469 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396c29e-dd79-4e52-9d6c-4dddfedb0777 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:57.717755 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396c29e-dd79-4e52-9d6c-4dddfedb0777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:57.718031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396c29e-dd79-4e52-9d6c-4dddfedb0777 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:57.718733 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7396c29e-dd79-4e52-9d6c-4dddfedb0777 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:57.720706 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7396c29e-dd79-4e52-9d6c-4dddfedb0777 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:57.793119 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 825/1670] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:30:57 2022] POST /identity/v3/users => generated 291 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:30:57.794345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116013c-cca0-4391-b52e-e2807f0d6072 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:57.813553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116013c-cca0-4391-b52e-e2807f0d6072 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:57.838605 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ffbf4-37e3-4f06-8ffe-5bb0933dfd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:57.839236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ffbf4-37e3-4f06-8ffe-5bb0933dfd3f admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:57.839517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ffbf4-37e3-4f06-8ffe-5bb0933dfd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:57.839788 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ffbf4-37e3-4f06-8ffe-5bb0933dfd3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:57.840591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5ffbf4-37e3-4f06-8ffe-5bb0933dfd3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:57.842789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5ffbf4-37e3-4f06-8ffe-5bb0933dfd3f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:57.893969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 846/1671] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:30:57.895293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396c29e-dd79-4e52-9d6c-4dddfedb0777 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:57.905615 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:57.927405 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396c29e-dd79-4e52-9d6c-4dddfedb0777 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:57.954414 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89bafbc-3a7f-438b-adea-3e8518af8bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:57.955058 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bafbc-3a7f-438b-adea-3e8518af8bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:57.955555 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bafbc-3a7f-438b-adea-3e8518af8bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:57.955849 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bafbc-3a7f-438b-adea-3e8518af8bf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:57.956511 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89bafbc-3a7f-438b-adea-3e8518af8bf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:57.958134 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89bafbc-3a7f-438b-adea-3e8518af8bf6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:58.011140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 826/1672] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:30:58.014329 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ffbf4-37e3-4f06-8ffe-5bb0933dfd3f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:58.037898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ffbf4-37e3-4f06-8ffe-5bb0933dfd3f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:58.067777 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1940e350-b852-48fa-9cb1-fd4dcd00efa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:58.068366 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940e350-b852-48fa-9cb1-fd4dcd00efa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:58.068804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940e350-b852-48fa-9cb1-fd4dcd00efa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:58.069213 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940e350-b852-48fa-9cb1-fd4dcd00efa5 admin admin] PATH_INFO: `/v3/projects/4aa90d955484485eb69a58c14dcfb18c/users/a13a334b8e874c739a91cc470f650d39/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:58.110520 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1940e350-b852-48fa-9cb1-fd4dcd00efa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aa90d955484485eb69a58c14dcfb18c, user_id=a13a334b8e874c739a91cc470f650d39, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:58.118733 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1940e350-b852-48fa-9cb1-fd4dcd00efa5 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:58.167579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 847/1673] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:30:58 2022] PUT /identity/v3/projects/4aa90d955484485eb69a58c14dcfb18c/users/a13a334b8e874c739a91cc470f650d39/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:30:58.182028 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd758a-f235-4bd8-83af-9832b7a29554 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:58.182420 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd758a-f235-4bd8-83af-9832b7a29554 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:58.182698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd758a-f235-4bd8-83af-9832b7a29554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:58.261009 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f0fd758a-f235-4bd8-83af-9832b7a29554 None None] MFA Rules not processed for user `a13a334b8e874c739a91cc470f650d39`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:58.264051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f0fd758a-f235-4bd8-83af-9832b7a29554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:58.441071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 827/1674] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:58.445499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1940e350-b852-48fa-9cb1-fd4dcd00efa5 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:58.510488 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:58.571294 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1940e350-b852-48fa-9cb1-fd4dcd00efa5 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:58.578953 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f139c6-d6ba-4a54-875a-1f1bface81de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:58.580163 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f139c6-d6ba-4a54-875a-1f1bface81de admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:58.580474 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f139c6-d6ba-4a54-875a-1f1bface81de admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:58.580776 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f139c6-d6ba-4a54-875a-1f1bface81de admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:58.581729 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f139c6-d6ba-4a54-875a-1f1bface81de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:58.583922 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f139c6-d6ba-4a54-875a-1f1bface81de admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:58.635576 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 848/1675] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:30:58 2022] POST /identity/v3/projects => generated 381 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:30:58.637562 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd758a-f235-4bd8-83af-9832b7a29554 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:58.656489 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd758a-f235-4bd8-83af-9832b7a29554 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:58.780121 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90445461-b4a1-42f0-8ed6-48078e6a69f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:58.781141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90445461-b4a1-42f0-8ed6-48078e6a69f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:58.781581 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90445461-b4a1-42f0-8ed6-48078e6a69f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:58.782010 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90445461-b4a1-42f0-8ed6-48078e6a69f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:58.782846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90445461-b4a1-42f0-8ed6-48078e6a69f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:58.785290 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90445461-b4a1-42f0-8ed6-48078e6a69f9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:58.815647 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 828/1676] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:30:58 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:30:58.820267 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f139c6-d6ba-4a54-875a-1f1bface81de admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:58.843990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bdc53-3f31-4665-9fdc-03702efc36a6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:58.845720 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bdc53-3f31-4665-9fdc-03702efc36a6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:58.846591 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bdc53-3f31-4665-9fdc-03702efc36a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:58.847935 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f139c6-d6ba-4a54-875a-1f1bface81de admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:58.860701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686014d9-b652-4402-a4e7-aa06cb3443af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:58.861501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686014d9-b652-4402-a4e7-aa06cb3443af admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:58.862232 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686014d9-b652-4402-a4e7-aa06cb3443af admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:58.862646 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686014d9-b652-4402-a4e7-aa06cb3443af admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:58.863458 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686014d9-b652-4402-a4e7-aa06cb3443af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:58.868642 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686014d9-b652-4402-a4e7-aa06cb3443af admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:58.904555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 849/1677] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:30:58.914556 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-124bdc53-3f31-4665-9fdc-03702efc36a6 None None] MFA Rules not processed for user `024adeb8a8e14d678ae14174d9555152`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:58.916601 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f144ff-e180-4d8a-9b71-977e73bd1713 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:58.917103 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f144ff-e180-4d8a-9b71-977e73bd1713 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:58.917785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f144ff-e180-4d8a-9b71-977e73bd1713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:58.923413 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-124bdc53-3f31-4665-9fdc-03702efc36a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:58.999221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f2f144ff-e180-4d8a-9b71-977e73bd1713 None None] MFA Rules not processed for user `0d4f8f3a6d42456d8e5cfbe20e54687a`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:59.004377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f2f144ff-e180-4d8a-9b71-977e73bd1713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:59.153020 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 829/1678] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:59.160691 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124bdc53-3f31-4665-9fdc-03702efc36a6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:59.168341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:59.185121 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124bdc53-3f31-4665-9fdc-03702efc36a6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:59.197664 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743f466-41a0-4abd-a1ad-2e71d8a288c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:59.198372 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743f466-41a0-4abd-a1ad-2e71d8a288c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:59.198669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743f466-41a0-4abd-a1ad-2e71d8a288c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:59.198930 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743f466-41a0-4abd-a1ad-2e71d8a288c3 admin admin] PATH_INFO: `/v3/projects/746e75634f284711b7050bc02c34afc9/users/0689a443cfcc493da740b95576642b39/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:59.223852 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 850/1679] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:59.225106 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f144ff-e180-4d8a-9b71-977e73bd1713 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:59.228272 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f2f144ff-e180-4d8a-9b71-977e73bd1713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:59.242462 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2743f466-41a0-4abd-a1ad-2e71d8a288c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=746e75634f284711b7050bc02c34afc9, user_id=0689a443cfcc493da740b95576642b39, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:59.245341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2743f466-41a0-4abd-a1ad-2e71d8a288c3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:59.272769 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 830/1680] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:30:59 2022] PUT /identity/v3/projects/746e75634f284711b7050bc02c34afc9/users/0689a443cfcc493da740b95576642b39/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:30:59.277407 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381cfcb-5c51-4085-9da8-a6dcf4a6c649 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:59.277648 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381cfcb-5c51-4085-9da8-a6dcf4a6c649 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:59.279652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381cfcb-5c51-4085-9da8-a6dcf4a6c649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:59.327996 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3381cfcb-5c51-4085-9da8-a6dcf4a6c649 None None] MFA Rules not processed for user `0d4f8f3a6d42456d8e5cfbe20e54687a`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:59.329228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3381cfcb-5c51-4085-9da8-a6dcf4a6c649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:59.400316 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f144ff-e180-4d8a-9b71-977e73bd1713 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:59.460549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 831/1681] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:59.461636 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381cfcb-5c51-4085-9da8-a6dcf4a6c649 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:59.465763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:59.571579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3ad12-e567-4f9b-bc21-e6ecbcaacc4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:59.572815 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3ad12-e567-4f9b-bc21-e6ecbcaacc4b service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:59.572815 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3ad12-e567-4f9b-bc21-e6ecbcaacc4b service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:59.573069 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3ad12-e567-4f9b-bc21-e6ecbcaacc4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:59.681790 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381cfcb-5c51-4085-9da8-a6dcf4a6c649 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:59.691087 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52efd5-b646-456d-a3c2-84b4a2748592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:59.691955 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52efd5-b646-456d-a3c2-84b4a2748592 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:59.692654 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52efd5-b646-456d-a3c2-84b4a2748592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:59.692935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52efd5-b646-456d-a3c2-84b4a2748592 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:59.693597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b52efd5-b646-456d-a3c2-84b4a2748592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:59.696664 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b52efd5-b646-456d-a3c2-84b4a2748592 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:59.736657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 832/1682] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:30:59.737645 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52efd5-b646-456d-a3c2-84b4a2748592 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:59.741768 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5b52efd5-b646-456d-a3c2-84b4a2748592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:59.771495 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee3ad12-e567-4f9b-bc21-e6ecbcaacc4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:59.773096 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee3ad12-e567-4f9b-bc21-e6ecbcaacc4b service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:59.781918 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52efd5-b646-456d-a3c2-84b4a2748592 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:59.796910 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e03ff-5b2b-4529-9faf-212096acd39f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:59.798270 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e03ff-5b2b-4529-9faf-212096acd39f service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:59.798270 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e03ff-5b2b-4529-9faf-212096acd39f service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:59.798452 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e03ff-5b2b-4529-9faf-212096acd39f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:59.801216 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 851/1683] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:59.802183 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3ad12-e567-4f9b-bc21-e6ecbcaacc4b service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:59.826703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e03ff-5b2b-4529-9faf-212096acd39f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:59.830453 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e03ff-5b2b-4529-9faf-212096acd39f service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:59.835665 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3ad12-e567-4f9b-bc21-e6ecbcaacc4b service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:59.849641 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6916b13-8ac6-481d-85d2-91160f395744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:59.850552 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6916b13-8ac6-481d-85d2-91160f395744 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:59.852749 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6916b13-8ac6-481d-85d2-91160f395744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:59.853173 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6916b13-8ac6-481d-85d2-91160f395744 admin admin] PATH_INFO: `/v3/projects/746e75634f284711b7050bc02c34afc9/users/0689a443cfcc493da740b95576642b39/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:59.855955 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 833/1684] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:59 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:59.858071 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6916b13-8ac6-481d-85d2-91160f395744 admin admin] RBAC: Authorizing `identity:create_grant(project_id=746e75634f284711b7050bc02c34afc9, user_id=0689a443cfcc493da740b95576642b39, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:59.861195 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6916b13-8ac6-481d-85d2-91160f395744 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:59.898634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 852/1685] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:30:59 2022] PUT /identity/v3/projects/746e75634f284711b7050bc02c34afc9/users/0689a443cfcc493da740b95576642b39/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:30:59.904322 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b36aeb-b3ff-46e6-9f53-c34213444aad None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:59.904606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b36aeb-b3ff-46e6-9f53-c34213444aad None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:59.904884 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b36aeb-b3ff-46e6-9f53-c34213444aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:59.960988 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c3b36aeb-b3ff-46e6-9f53-c34213444aad None None] MFA Rules not processed for user `0689a443cfcc493da740b95576642b39`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:59.962959 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c3b36aeb-b3ff-46e6-9f53-c34213444aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:00.148944 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:00.207089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 834/1686] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:30:59 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:31:00.219068 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6916b13-8ac6-481d-85d2-91160f395744 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:00.220943 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b6916b13-8ac6-481d-85d2-91160f395744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:00.378237 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6916b13-8ac6-481d-85d2-91160f395744 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:00.393033 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83188f51-4ff1-4e49-b655-12fbb2dd7a43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:00.393667 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83188f51-4ff1-4e49-b655-12fbb2dd7a43 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:00.393948 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83188f51-4ff1-4e49-b655-12fbb2dd7a43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:00.394650 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83188f51-4ff1-4e49-b655-12fbb2dd7a43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:00.424621 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83188f51-4ff1-4e49-b655-12fbb2dd7a43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:00.426079 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83188f51-4ff1-4e49-b655-12fbb2dd7a43 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:00.446779 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 853/1687] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:00 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:00.891186 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b33df7-5113-4777-9a15-d8e8631fc70d None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:00.891477 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b33df7-5113-4777-9a15-d8e8631fc70d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:00.891809 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b33df7-5113-4777-9a15-d8e8631fc70d None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:00.894139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 835/1688] 10.222.0.58 () {54 vars in 912 bytes} [Thu Sep 1 16:31:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:31:00.915820 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba427793-b19f-4832-96cc-7d3d8256d3df None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:00.916301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba427793-b19f-4832-96cc-7d3d8256d3df None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:00.917215 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba427793-b19f-4832-96cc-7d3d8256d3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:00.918282 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b33df7-5113-4777-9a15-d8e8631fc70d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:00.920274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-42b33df7-5113-4777-9a15-d8e8631fc70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:00.940641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b33df7-5113-4777-9a15-d8e8631fc70d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:00.946091 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635de792-be5a-489b-a60a-1a68fb85b0c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:00.948124 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635de792-be5a-489b-a60a-1a68fb85b0c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:00.948629 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635de792-be5a-489b-a60a-1a68fb85b0c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:00.949046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635de792-be5a-489b-a60a-1a68fb85b0c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:01.045742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635de792-be5a-489b-a60a-1a68fb85b0c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:01.050404 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635de792-be5a-489b-a60a-1a68fb85b0c8 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:01.074950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 836/1689] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:01.080670 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ba427793-b19f-4832-96cc-7d3d8256d3df None None] MFA Rules not processed for user `943fae29c5dc41f2911bbe051c74a361`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:01.082569 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ba427793-b19f-4832-96cc-7d3d8256d3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:01.199649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:01.256892 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 854/1690] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Sep 1 16:31:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:01.270585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635de792-be5a-489b-a60a-1a68fb85b0c8 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:01.272365 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-635de792-be5a-489b-a60a-1a68fb85b0c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:01.292836 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635de792-be5a-489b-a60a-1a68fb85b0c8 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:01.304986 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef99a281-afc4-402f-b884-d91fddd3683c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:01.305798 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99a281-afc4-402f-b884-d91fddd3683c service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:01.306591 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99a281-afc4-402f-b884-d91fddd3683c service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:01.307857 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99a281-afc4-402f-b884-d91fddd3683c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:01.326794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef99a281-afc4-402f-b884-d91fddd3683c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:01.328830 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef99a281-afc4-402f-b884-d91fddd3683c service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:01.359404 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 837/1691] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:31:01.469968 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d1fa7-46d1-4ef5-ac67-390b1655c56d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:01.470445 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d1fa7-46d1-4ef5-ac67-390b1655c56d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:01.470796 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d1fa7-46d1-4ef5-ac67-390b1655c56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:01.564575 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-3d5d1fa7-46d1-4ef5-ac67-390b1655c56d None None] MFA Rules not processed for user `a328e946d6d4441ca7ecbb3a20d2a611`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:01.565802 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3d5d1fa7-46d1-4ef5-ac67-390b1655c56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:01.700858 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 855/1692] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:01 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:01.717096 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef99a281-afc4-402f-b884-d91fddd3683c service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:01.913614 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef99a281-afc4-402f-b884-d91fddd3683c service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:01.932761 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb54d2-19fb-4850-94de-879c252e9726 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:01.933540 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb54d2-19fb-4850-94de-879c252e9726 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:01.933825 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb54d2-19fb-4850-94de-879c252e9726 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:01.934101 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb54d2-19fb-4850-94de-879c252e9726 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:01.962849 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccb54d2-19fb-4850-94de-879c252e9726 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:01.968575 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccb54d2-19fb-4850-94de-879c252e9726 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:02.012995 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 838/1693] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:01 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:02.292054 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d1fa7-46d1-4ef5-ac67-390b1655c56d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:02.293696 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3d5d1fa7-46d1-4ef5-ac67-390b1655c56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:02.367995 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb54d2-19fb-4850-94de-879c252e9726 service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:02.369553 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dccb54d2-19fb-4850-94de-879c252e9726 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:02.467418 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d1fa7-46d1-4ef5-ac67-390b1655c56d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:02.482087 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b2ef3-c291-4603-9048-57b3b4e7f03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:02.483260 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b2ef3-c291-4603-9048-57b3b4e7f03e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:02.483876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b2ef3-c291-4603-9048-57b3b4e7f03e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:02.483876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b2ef3-c291-4603-9048-57b3b4e7f03e admin admin] PATH_INFO: `/v3/users/a0868f941e5e4d2184ab95316f7075d9` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:02.488209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b2ef3-c291-4603-9048-57b3b4e7f03e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0868f941e5e4d2184ab95316f7075d9)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:02.490162 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b2ef3-c291-4603-9048-57b3b4e7f03e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:02.519414 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:02.530685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb54d2-19fb-4850-94de-879c252e9726 service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:02.538830 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc64de-c45e-4e6d-9216-f59d906204be service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12302cdb2854b0684179647a6dadf6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:02.540139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc64de-c45e-4e6d-9216-f59d906204be service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:02.540540 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc64de-c45e-4e6d-9216-f59d906204be service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:02.541726 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc64de-c45e-4e6d-9216-f59d906204be service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:02.543517 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-89fc64de-c45e-4e6d-9216-f59d906204be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:02.558339 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fc64de-c45e-4e6d-9216-f59d906204be service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:02.561328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fc64de-c45e-4e6d-9216-f59d906204be service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:02.577163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 839/1694] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:02.622329 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3d687-822c-4ed7-99d9-b52ff3ff5298 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:02.623282 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3d687-822c-4ed7-99d9-b52ff3ff5298 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:02.623907 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3d687-822c-4ed7-99d9-b52ff3ff5298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:02.695017 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ff3b2ef3-c291-4603-9048-57b3b4e7f03e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0868f941e5e4d2184ab95316f7075d9'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:02.704039 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-54c3d687-822c-4ed7-99d9-b52ff3ff5298 None None] MFA Rules not processed for user `0d4f8f3a6d42456d8e5cfbe20e54687a`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:02.706057 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-54c3d687-822c-4ed7-99d9-b52ff3ff5298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:02.724284 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ff3b2ef3-c291-4603-9048-57b3b4e7f03e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0868f941e5e4d2184ab95316f7075d9'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:02.727589 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ff3b2ef3-c291-4603-9048-57b3b4e7f03e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0868f941e5e4d2184ab95316f7075d9'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:02.730647 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ff3b2ef3-c291-4603-9048-57b3b4e7f03e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0868f941e5e4d2184ab95316f7075d9'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:02.772796 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ff3b2ef3-c291-4603-9048-57b3b4e7f03e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0868f941e5e4d2184ab95316f7075d9'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:02.789518 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:02.811316 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 856/1695] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:31:02 2022] DELETE /identity/v3/users/a0868f941e5e4d2184ab95316f7075d9 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:02.813972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d363ef1-8402-45d5-ad2b-b3cbdb39bdea None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:02.814453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d363ef1-8402-45d5-ad2b-b3cbdb39bdea None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:02.814735 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d363ef1-8402-45d5-ad2b-b3cbdb39bdea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:02.875257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-3d363ef1-8402-45d5-ad2b-b3cbdb39bdea None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:02.882800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3d363ef1-8402-45d5-ad2b-b3cbdb39bdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:02.918290 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 840/1696] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:03.146377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 857/1697] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:03.178600 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3d687-822c-4ed7-99d9-b52ff3ff5298 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:03.180547 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-54c3d687-822c-4ed7-99d9-b52ff3ff5298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:03.208762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3d687-822c-4ed7-99d9-b52ff3ff5298 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:03.320685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffaa522-7e71-4921-a05e-65f7df8ead4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:03.321922 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffaa522-7e71-4921-a05e-65f7df8ead4f service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:03.322237 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffaa522-7e71-4921-a05e-65f7df8ead4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:03.322497 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffaa522-7e71-4921-a05e-65f7df8ead4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:03.336514 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffaa522-7e71-4921-a05e-65f7df8ead4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:03.339829 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffaa522-7e71-4921-a05e-65f7df8ead4f service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:03.359862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 841/1698] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:03.834574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b0313-c1fc-48d4-b3e0-1933bc1220da None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:03.835083 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b0313-c1fc-48d4-b3e0-1933bc1220da None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:03.835299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b0313-c1fc-48d4-b3e0-1933bc1220da None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:03.836817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 858/1699] 10.222.0.58 () {54 vars in 912 bytes} [Thu Sep 1 16:31:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:31:03.847764 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acb07a-2305-4b1a-8224-6eafb1643b4f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:03.848636 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acb07a-2305-4b1a-8224-6eafb1643b4f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:03.849362 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acb07a-2305-4b1a-8224-6eafb1643b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:03.904993 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a7acb07a-2305-4b1a-8224-6eafb1643b4f None None] MFA Rules not processed for user `943fae29c5dc41f2911bbe051c74a361`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:03.906881 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a7acb07a-2305-4b1a-8224-6eafb1643b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:03.919503 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 842/1700] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Sep 1 16:31:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:03.963233 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b0313-c1fc-48d4-b3e0-1933bc1220da None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:03.982859 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b0313-c1fc-48d4-b3e0-1933bc1220da None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:03.990286 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a10bd57-fd64-4e8f-b496-a8f3e3f6c199 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:03.991003 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10bd57-fd64-4e8f-b496-a8f3e3f6c199 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:03.991337 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10bd57-fd64-4e8f-b496-a8f3e3f6c199 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:03.991619 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10bd57-fd64-4e8f-b496-a8f3e3f6c199 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:04.026843 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a10bd57-fd64-4e8f-b496-a8f3e3f6c199 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:04.028592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a10bd57-fd64-4e8f-b496-a8f3e3f6c199 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:04.042350 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 859/1701] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:04.432373 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8f5bc-ae87-4c16-b635-079e581577d6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:04.432658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8f5bc-ae87-4c16-b635-079e581577d6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:04.432933 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8f5bc-ae87-4c16-b635-079e581577d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:04.516214 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-63d8f5bc-ae87-4c16-b635-079e581577d6 None None] MFA Rules not processed for user `f69ac454b5a64e6d83188ad2b8e107d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:04.517638 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-63d8f5bc-ae87-4c16-b635-079e581577d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:04.586227 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a10bd57-fd64-4e8f-b496-a8f3e3f6c199 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:04.586227 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3a10bd57-fd64-4e8f-b496-a8f3e3f6c199 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:04.642167 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a10bd57-fd64-4e8f-b496-a8f3e3f6c199 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:04.668739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d61102-2d1c-4389-8d07-b8ca79bcf9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:04.669632 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d61102-2d1c-4389-8d07-b8ca79bcf9b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:04.669716 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d61102-2d1c-4389-8d07-b8ca79bcf9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:04.670075 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d61102-2d1c-4389-8d07-b8ca79bcf9b6 admin admin] PATH_INFO: `/v3/users/a13a334b8e874c739a91cc470f650d39` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:04.675065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d61102-2d1c-4389-8d07-b8ca79bcf9b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a13a334b8e874c739a91cc470f650d39)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:04.677033 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d61102-2d1c-4389-8d07-b8ca79bcf9b6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:04.818334 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 843/1702] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:04.830614 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d06fa9-b2e7-4b76-9cf8-93e0a2dd1f2f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:04.830614 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d06fa9-b2e7-4b76-9cf8-93e0a2dd1f2f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:04.830860 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d06fa9-b2e7-4b76-9cf8-93e0a2dd1f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:04.839141 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-34d61102-2d1c-4389-8d07-b8ca79bcf9b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a13a334b8e874c739a91cc470f650d39'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:04.855963 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-34d61102-2d1c-4389-8d07-b8ca79bcf9b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a13a334b8e874c739a91cc470f650d39'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:04.859060 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-34d61102-2d1c-4389-8d07-b8ca79bcf9b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a13a334b8e874c739a91cc470f650d39'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:04.863647 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-34d61102-2d1c-4389-8d07-b8ca79bcf9b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a13a334b8e874c739a91cc470f650d39'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:04.884371 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-22d06fa9-b2e7-4b76-9cf8-93e0a2dd1f2f None None] MFA Rules not processed for user `f69ac454b5a64e6d83188ad2b8e107d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:04.888191 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-22d06fa9-b2e7-4b76-9cf8-93e0a2dd1f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:04.905934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-34d61102-2d1c-4389-8d07-b8ca79bcf9b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a13a334b8e874c739a91cc470f650d39'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:04.917034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 844/1703] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:04.917686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d06fa9-b2e7-4b76-9cf8-93e0a2dd1f2f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:04.923626 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-22d06fa9-b2e7-4b76-9cf8-93e0a2dd1f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:04.945295 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 860/1704] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:31:04 2022] DELETE /identity/v3/users/a13a334b8e874c739a91cc470f650d39 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:31:04.949344 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d61102-2d1c-4389-8d07-b8ca79bcf9b6 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:04.951603 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-34d61102-2d1c-4389-8d07-b8ca79bcf9b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:04.986341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d06fa9-b2e7-4b76-9cf8-93e0a2dd1f2f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:05.036016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:05.216256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:05.221492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f972a-4c68-4a0f-8486-cde57ce29d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:05.222974 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f972a-4c68-4a0f-8486-cde57ce29d34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:05.223282 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f972a-4c68-4a0f-8486-cde57ce29d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:05.223568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f972a-4c68-4a0f-8486-cde57ce29d34 admin admin] PATH_INFO: `/v3/projects/b62e057f688142fba6a242a424bd4522` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:05.224828 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f972a-4c68-4a0f-8486-cde57ce29d34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b62e057f688142fba6a242a424bd4522)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:05.234728 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f972a-4c68-4a0f-8486-cde57ce29d34 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:05.283522 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d61102-2d1c-4389-8d07-b8ca79bcf9b6 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:05.293393 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5240850-babd-46ed-a8b8-30d3aa0c1333 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:05.294225 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5240850-babd-46ed-a8b8-30d3aa0c1333 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:05.295232 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5240850-babd-46ed-a8b8-30d3aa0c1333 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:05.295655 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5240850-babd-46ed-a8b8-30d3aa0c1333 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:05.301463 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e5240850-babd-46ed-a8b8-30d3aa0c1333 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:05.346063 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3d5f972a-4c68-4a0f-8486-cde57ce29d34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b62e057f688142fba6a242a424bd4522', 'request_id': 'req-3d5f972a-4c68-4a0f-8486-cde57ce29d34'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:05.358938 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3d5f972a-4c68-4a0f-8486-cde57ce29d34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b62e057f688142fba6a242a424bd4522', 'request_id': 'req-3d5f972a-4c68-4a0f-8486-cde57ce29d34'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:05.377860 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3d5f972a-4c68-4a0f-8486-cde57ce29d34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b62e057f688142fba6a242a424bd4522', 'request_id': 'req-3d5f972a-4c68-4a0f-8486-cde57ce29d34'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:05.414935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3d5f972a-4c68-4a0f-8486-cde57ce29d34 admin admin] The token cache is being invalidate because project b62e057f688142fba6a242a424bd4522 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:31:05.415147 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3d5f972a-4c68-4a0f-8486-cde57ce29d34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:05.420502 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3d5f972a-4c68-4a0f-8486-cde57ce29d34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:05.423721 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 845/1705] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:31:04 2022] DELETE /identity/v3/projects/b62e057f688142fba6a242a424bd4522 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:05.429729 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab0448-33db-4596-97d8-319592de8276 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:05.430022 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab0448-33db-4596-97d8-319592de8276 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:05.430331 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab0448-33db-4596-97d8-319592de8276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:05.451366 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5240850-babd-46ed-a8b8-30d3aa0c1333 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:05.455238 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5240850-babd-46ed-a8b8-30d3aa0c1333 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:05.477957 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b7ab0448-33db-4596-97d8-319592de8276 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:05.482825 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b7ab0448-33db-4596-97d8-319592de8276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:05.487406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 861/1706] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:05.491552 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5240850-babd-46ed-a8b8-30d3aa0c1333 service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:05.536279 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5240850-babd-46ed-a8b8-30d3aa0c1333 service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:05.744951 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 846/1707] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:05.770689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9071c-6fa3-4aec-a2e0-5384364e62b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:05.771354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9071c-6fa3-4aec-a2e0-5384364e62b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:05.771837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9071c-6fa3-4aec-a2e0-5384364e62b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:05.772369 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9071c-6fa3-4aec-a2e0-5384364e62b1 admin admin] PATH_INFO: `/v3/users/41611648338f4b2192bd373a49e8431b` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:05.774414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a9071c-6fa3-4aec-a2e0-5384364e62b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41611648338f4b2192bd373a49e8431b)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:05.776372 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a9071c-6fa3-4aec-a2e0-5384364e62b1 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:05.850051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab0448-33db-4596-97d8-319592de8276 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:05.851652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b7ab0448-33db-4596-97d8-319592de8276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:05.964356 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-55a9071c-6fa3-4aec-a2e0-5384364e62b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41611648338f4b2192bd373a49e8431b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:06.021014 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-55a9071c-6fa3-4aec-a2e0-5384364e62b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41611648338f4b2192bd373a49e8431b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:06.022770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-55a9071c-6fa3-4aec-a2e0-5384364e62b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41611648338f4b2192bd373a49e8431b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:06.023675 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-55a9071c-6fa3-4aec-a2e0-5384364e62b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41611648338f4b2192bd373a49e8431b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:06.050986 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:06.068930 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-55a9071c-6fa3-4aec-a2e0-5384364e62b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41611648338f4b2192bd373a49e8431b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:06.075742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab0448-33db-4596-97d8-319592de8276 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:06.089375 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a8a32-174d-4fe1-ba01-c4061606f6a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:06.090006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a8a32-174d-4fe1-ba01-c4061606f6a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:06.090327 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a8a32-174d-4fe1-ba01-c4061606f6a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:06.090606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a8a32-174d-4fe1-ba01-c4061606f6a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:06.115684 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-479a8a32-174d-4fe1-ba01-c4061606f6a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:06.119971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 862/1708] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:31:05 2022] DELETE /identity/v3/users/41611648338f4b2192bd373a49e8431b => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:06.139974 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a8a32-174d-4fe1-ba01-c4061606f6a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:06.141366 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a8a32-174d-4fe1-ba01-c4061606f6a0 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:06.179108 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 847/1709] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:06.727734 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427d455-fa57-457b-961a-04b0f7b9e146 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:06.728263 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427d455-fa57-457b-961a-04b0f7b9e146 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:06.728979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427d455-fa57-457b-961a-04b0f7b9e146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:06.847279 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:06.852441 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2427d455-fa57-457b-961a-04b0f7b9e146 None None] MFA Rules not processed for user `4ecfc51400414558b8389a9c35b2fa1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:06.852441 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2427d455-fa57-457b-961a-04b0f7b9e146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:06.894295 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b9c3d-af71-4812-8e78-93fd7831b5ac None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:06.894837 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b9c3d-af71-4812-8e78-93fd7831b5ac None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:06.895334 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b9c3d-af71-4812-8e78-93fd7831b5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:07.040207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 863/1710] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:07.129430 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b24b9c3d-af71-4812-8e78-93fd7831b5ac None None] MFA Rules not processed for user `dd7b98536fa445cebd552fcbf54b757d`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:07.133157 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b24b9c3d-af71-4812-8e78-93fd7831b5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:07.310668 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 848/1711] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Sep 1 16:31:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:31:07.489895 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427d455-fa57-457b-961a-04b0f7b9e146 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:07.491369 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2427d455-fa57-457b-961a-04b0f7b9e146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:07.640927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b9c3d-af71-4812-8e78-93fd7831b5ac None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:07.642802 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b24b9c3d-af71-4812-8e78-93fd7831b5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:07.705309 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427d455-fa57-457b-961a-04b0f7b9e146 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:07.722850 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c9bb4c-7d1f-474b-8668-5d8699443fc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:07.725454 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9bb4c-7d1f-474b-8668-5d8699443fc2 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:07.728570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9bb4c-7d1f-474b-8668-5d8699443fc2 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:07.728807 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9bb4c-7d1f-474b-8668-5d8699443fc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:07.750504 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c9bb4c-7d1f-474b-8668-5d8699443fc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:07.754123 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c9bb4c-7d1f-474b-8668-5d8699443fc2 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:07.805613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 864/1712] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:07.903107 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24126563-0c32-4732-822c-fb6a7e3dc9d2 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:07.904609 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24126563-0c32-4732-822c-fb6a7e3dc9d2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:07.905305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24126563-0c32-4732-822c-fb6a7e3dc9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:07.953533 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-24126563-0c32-4732-822c-fb6a7e3dc9d2 None None] MFA Rules not processed for user `024adeb8a8e14d678ae14174d9555152`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:07.956438 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-24126563-0c32-4732-822c-fb6a7e3dc9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:07.979698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b9c3d-af71-4812-8e78-93fd7831b5ac None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:07.987255 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40366ef5-42c6-4b25-aef5-fbf511829e98 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e789695517c1437fb224f5a6195655f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:07.987688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40366ef5-42c6-4b25-aef5-fbf511829e98 service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:07.988122 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40366ef5-42c6-4b25-aef5-fbf511829e98 service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:07.988621 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40366ef5-42c6-4b25-aef5-fbf511829e98 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:08.009067 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40366ef5-42c6-4b25-aef5-fbf511829e98 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:08.012532 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40366ef5-42c6-4b25-aef5-fbf511829e98 service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:08.094225 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 849/1713] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Sep 1 16:31:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:31:08.143451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 865/1714] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:08.299139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40366ef5-42c6-4b25-aef5-fbf511829e98 service swift] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:08.300725 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-40366ef5-42c6-4b25-aef5-fbf511829e98 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:08.474567 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:08.605367 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40366ef5-42c6-4b25-aef5-fbf511829e98 service swift] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:08.629327 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e4fe3-865d-4f52-a105-35a0134db116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:08.630517 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e4fe3-865d-4f52-a105-35a0134db116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:08.630916 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e4fe3-865d-4f52-a105-35a0134db116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:08.631304 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e4fe3-865d-4f52-a105-35a0134db116 admin admin] PATH_INFO: `/v3/users/8bf3787ee7ab4063b3edff49ab345369` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:08.637113 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e4fe3-865d-4f52-a105-35a0134db116 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bf3787ee7ab4063b3edff49ab345369)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:08.643737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e4fe3-865d-4f52-a105-35a0134db116 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:08.775022 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-366e4fe3-865d-4f52-a105-35a0134db116 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bf3787ee7ab4063b3edff49ab345369'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:08.821821 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-366e4fe3-865d-4f52-a105-35a0134db116 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bf3787ee7ab4063b3edff49ab345369'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:08.823488 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-366e4fe3-865d-4f52-a105-35a0134db116 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bf3787ee7ab4063b3edff49ab345369'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:08.870640 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-366e4fe3-865d-4f52-a105-35a0134db116 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bf3787ee7ab4063b3edff49ab345369'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:08.945206 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-366e4fe3-865d-4f52-a105-35a0134db116 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bf3787ee7ab4063b3edff49ab345369'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:08.986702 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2580188-bfd6-406b-8f50-e88ff55f1d12 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:08.986990 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2580188-bfd6-406b-8f50-e88ff55f1d12 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:08.987285 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2580188-bfd6-406b-8f50-e88ff55f1d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:09.002303 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 850/1715] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:31:08 2022] DELETE /identity/v3/users/8bf3787ee7ab4063b3edff49ab345369 => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:09.024546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a2580188-bfd6-406b-8f50-e88ff55f1d12 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:09.025979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a2580188-bfd6-406b-8f50-e88ff55f1d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:09.098751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:09.169254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 866/1716] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:09.189631 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e4fe3-865d-4f52-a105-35a0134db116 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:09.191297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-366e4fe3-865d-4f52-a105-35a0134db116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:09.220998 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e4fe3-865d-4f52-a105-35a0134db116 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:09.249313 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f387735-4678-430b-a503-b0ccd839510f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:09.251083 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f387735-4678-430b-a503-b0ccd839510f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:09.253397 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f387735-4678-430b-a503-b0ccd839510f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:09.253862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f387735-4678-430b-a503-b0ccd839510f admin admin] PATH_INFO: `/v3/projects/1a7a7f7b15b84a47a9cac1a2520e442f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:09.258986 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f387735-4678-430b-a503-b0ccd839510f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a7a7f7b15b84a47a9cac1a2520e442f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:09.262601 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f387735-4678-430b-a503-b0ccd839510f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:09.392398 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2580188-bfd6-406b-8f50-e88ff55f1d12 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:09.394398 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a2580188-bfd6-406b-8f50-e88ff55f1d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:09.449164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9f387735-4678-430b-a503-b0ccd839510f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a7a7f7b15b84a47a9cac1a2520e442f', 'request_id': 'req-9f387735-4678-430b-a503-b0ccd839510f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:09.463781 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2580188-bfd6-406b-8f50-e88ff55f1d12 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:09.466903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9f387735-4678-430b-a503-b0ccd839510f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a7a7f7b15b84a47a9cac1a2520e442f', 'request_id': 'req-9f387735-4678-430b-a503-b0ccd839510f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:09.486894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9f387735-4678-430b-a503-b0ccd839510f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a7a7f7b15b84a47a9cac1a2520e442f', 'request_id': 'req-9f387735-4678-430b-a503-b0ccd839510f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:09.506554 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9f387735-4678-430b-a503-b0ccd839510f admin admin] The token cache is being invalidate because project 1a7a7f7b15b84a47a9cac1a2520e442f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:31:09.506980 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9f387735-4678-430b-a503-b0ccd839510f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:09.509015 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9f387735-4678-430b-a503-b0ccd839510f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:09.514829 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 851/1717] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:31:09 2022] DELETE /identity/v3/projects/1a7a7f7b15b84a47a9cac1a2520e442f => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:09.516501 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c66fb-24ea-4987-9856-8e627ec4b30c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:09.517349 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c66fb-24ea-4987-9856-8e627ec4b30c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:09.517349 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c66fb-24ea-4987-9856-8e627ec4b30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:09.563732 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-195c66fb-24ea-4987-9856-8e627ec4b30c None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:09.566750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-195c66fb-24ea-4987-9856-8e627ec4b30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:09.685908 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:09.760706 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebbf375-40cb-4815-a6e1-a469e3a48e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:09.780826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbf375-40cb-4815-a6e1-a469e3a48e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:09.781116 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbf375-40cb-4815-a6e1-a469e3a48e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:09.781395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbf375-40cb-4815-a6e1-a469e3a48e50 admin admin] PATH_INFO: `/v3/projects/4aa90d955484485eb69a58c14dcfb18c` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:09.790876 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 852/1718] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:09.792077 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c66fb-24ea-4987-9856-8e627ec4b30c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:09.793657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-195c66fb-24ea-4987-9856-8e627ec4b30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:09.797683 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebbf375-40cb-4815-a6e1-a469e3a48e50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4aa90d955484485eb69a58c14dcfb18c)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:09.801963 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebbf375-40cb-4815-a6e1-a469e3a48e50 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:10.040679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c66fb-24ea-4987-9856-8e627ec4b30c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:10.098354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-eebbf375-40cb-4815-a6e1-a469e3a48e50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4aa90d955484485eb69a58c14dcfb18c', 'request_id': 'req-eebbf375-40cb-4815-a6e1-a469e3a48e50'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:10.133219 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-eebbf375-40cb-4815-a6e1-a469e3a48e50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4aa90d955484485eb69a58c14dcfb18c', 'request_id': 'req-eebbf375-40cb-4815-a6e1-a469e3a48e50'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:10.158793 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-eebbf375-40cb-4815-a6e1-a469e3a48e50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4aa90d955484485eb69a58c14dcfb18c', 'request_id': 'req-eebbf375-40cb-4815-a6e1-a469e3a48e50'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:10.208337 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-eebbf375-40cb-4815-a6e1-a469e3a48e50 admin admin] The token cache is being invalidate because project 4aa90d955484485eb69a58c14dcfb18c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:31:10.209040 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-eebbf375-40cb-4815-a6e1-a469e3a48e50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:10.212318 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-eebbf375-40cb-4815-a6e1-a469e3a48e50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:10.219014 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 867/1719] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:31:09 2022] DELETE /identity/v3/projects/4aa90d955484485eb69a58c14dcfb18c => generated 0 bytes in 827 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:10.222182 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b83bc0-6571-4b30-b998-9eedacc1f23d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:10.223588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b83bc0-6571-4b30-b998-9eedacc1f23d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:10.224909 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b83bc0-6571-4b30-b998-9eedacc1f23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:10.241115 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05d5d3-0e0c-4782-9b06-c703c99693ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:10.242030 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05d5d3-0e0c-4782-9b06-c703c99693ab service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:10.242486 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05d5d3-0e0c-4782-9b06-c703c99693ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:10.242902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05d5d3-0e0c-4782-9b06-c703c99693ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:10.260553 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bc05d5d3-0e0c-4782-9b06-c703c99693ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:10.280443 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-58b83bc0-6571-4b30-b998-9eedacc1f23d None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:10.283163 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-58b83bc0-6571-4b30-b998-9eedacc1f23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:10.403575 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 868/1720] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:10.408123 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b83bc0-6571-4b30-b998-9eedacc1f23d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:10.412158 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-58b83bc0-6571-4b30-b998-9eedacc1f23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:10.427343 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:10.432902 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b83bc0-6571-4b30-b998-9eedacc1f23d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:10.441940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fde00a-c762-4bd1-bca4-b4d82798a139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:10.442949 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fde00a-c762-4bd1-bca4-b4d82798a139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:10.443467 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fde00a-c762-4bd1-bca4-b4d82798a139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:10.444401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fde00a-c762-4bd1-bca4-b4d82798a139 admin admin] PATH_INFO: `/v3/users/0689a443cfcc493da740b95576642b39` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:10.446488 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fde00a-c762-4bd1-bca4-b4d82798a139 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0689a443cfcc493da740b95576642b39)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:10.449268 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fde00a-c762-4bd1-bca4-b4d82798a139 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:10.475656 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05d5d3-0e0c-4782-9b06-c703c99693ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:10.477674 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05d5d3-0e0c-4782-9b06-c703c99693ab service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:10.495026 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 853/1721] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 703 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:10.499056 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324cf91-6d5d-4c48-b752-c7112d0a5f02 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:10.499348 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324cf91-6d5d-4c48-b752-c7112d0a5f02 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:10.499629 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324cf91-6d5d-4c48-b752-c7112d0a5f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:10.537795 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9324cf91-6d5d-4c48-b752-c7112d0a5f02 None None] MFA Rules not processed for user `a328e946d6d4441ca7ecbb3a20d2a611`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:10.540409 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9324cf91-6d5d-4c48-b752-c7112d0a5f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:10.593406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b1fde00a-c762-4bd1-bca4-b4d82798a139 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0689a443cfcc493da740b95576642b39'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:10.614068 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b1fde00a-c762-4bd1-bca4-b4d82798a139 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0689a443cfcc493da740b95576642b39'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:10.615250 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b1fde00a-c762-4bd1-bca4-b4d82798a139 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0689a443cfcc493da740b95576642b39'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:10.616392 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b1fde00a-c762-4bd1-bca4-b4d82798a139 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0689a443cfcc493da740b95576642b39'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:10.648342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:10.691362 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b1fde00a-c762-4bd1-bca4-b4d82798a139 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0689a443cfcc493da740b95576642b39'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:10.713069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 854/1722] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:10.714340 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324cf91-6d5d-4c48-b752-c7112d0a5f02 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:10.717267 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9324cf91-6d5d-4c48-b752-c7112d0a5f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:10.734837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 869/1723] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:31:10 2022] DELETE /identity/v3/users/0689a443cfcc493da740b95576642b39 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:10.752837 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324cf91-6d5d-4c48-b752-c7112d0a5f02 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:10.940455 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a011d5-cf37-4da8-9323-0d4f5953ada5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:10.941368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a011d5-cf37-4da8-9323-0d4f5953ada5 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:10.941801 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a011d5-cf37-4da8-9323-0d4f5953ada5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:10.942254 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a011d5-cf37-4da8-9323-0d4f5953ada5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:10.943076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a011d5-cf37-4da8-9323-0d4f5953ada5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:31:10.943527 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a011d5-cf37-4da8-9323-0d4f5953ada5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:10.945353 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a011d5-cf37-4da8-9323-0d4f5953ada5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:10.972589 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 855/1724] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:31:10.978047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fde00a-c762-4bd1-bca4-b4d82798a139 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:11.002888 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fde00a-c762-4bd1-bca4-b4d82798a139 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:11.027286 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e3db3c-61d1-4fe6-9008-a0cbe9db3a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:11.028181 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3db3c-61d1-4fe6-9008-a0cbe9db3a36 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:11.029214 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3db3c-61d1-4fe6-9008-a0cbe9db3a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:11.029642 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3db3c-61d1-4fe6-9008-a0cbe9db3a36 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:11.030556 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e3db3c-61d1-4fe6-9008-a0cbe9db3a36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:11.033205 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e3db3c-61d1-4fe6-9008-a0cbe9db3a36 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:11.150157 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 870/1725] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:31:10 2022] POST /identity/v3/projects => generated 363 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:11.155221 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a011d5-cf37-4da8-9323-0d4f5953ada5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:11.169723 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a011d5-cf37-4da8-9323-0d4f5953ada5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:11.362718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae775a-144d-44a2-a52f-2835a965e112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:11.364561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae775a-144d-44a2-a52f-2835a965e112 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:11.365107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae775a-144d-44a2-a52f-2835a965e112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:11.365679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae775a-144d-44a2-a52f-2835a965e112 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:11.366705 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae775a-144d-44a2-a52f-2835a965e112 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:11.381454 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae775a-144d-44a2-a52f-2835a965e112 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:11.435296 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e3db3c-61d1-4fe6-9008-a0cbe9db3a36 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:11.439753 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 856/1726] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:31:11 2022] POST /identity/v3/users => generated 282 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:11.471737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e3db3c-61d1-4fe6-9008-a0cbe9db3a36 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:11.492085 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91bd30-4430-491e-ab79-d303a5b6c021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:11.492733 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91bd30-4430-491e-ab79-d303a5b6c021 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:11.493104 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91bd30-4430-491e-ab79-d303a5b6c021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:11.493288 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91bd30-4430-491e-ab79-d303a5b6c021 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:11.493954 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91bd30-4430-491e-ab79-d303a5b6c021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:11.498747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91bd30-4430-491e-ab79-d303a5b6c021 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:11.536331 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 871/1727] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:31:11.541795 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae775a-144d-44a2-a52f-2835a965e112 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:11.577543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae775a-144d-44a2-a52f-2835a965e112 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:11.598638 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5513259c-ada3-4f08-a6ba-ba9bd6c16e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:11.601050 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5513259c-ada3-4f08-a6ba-ba9bd6c16e86 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:11.601445 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5513259c-ada3-4f08-a6ba-ba9bd6c16e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:11.602022 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5513259c-ada3-4f08-a6ba-ba9bd6c16e86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:11.603031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5513259c-ada3-4f08-a6ba-ba9bd6c16e86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:11.606014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5513259c-ada3-4f08-a6ba-ba9bd6c16e86 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:11.648822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 857/1728] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:11.650721 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91bd30-4430-491e-ab79-d303a5b6c021 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:11.672629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91bd30-4430-491e-ab79-d303a5b6c021 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:11.692822 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb671ef-b4c2-4255-b3b2-29c78510ab85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:11.693517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb671ef-b4c2-4255-b3b2-29c78510ab85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:11.693804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb671ef-b4c2-4255-b3b2-29c78510ab85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:11.694510 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb671ef-b4c2-4255-b3b2-29c78510ab85 admin admin] PATH_INFO: `/v3/projects/6c81f9beca474b8fb7a5fec365cb2fca/users/b5edd6fd393540838f1e3a171305f647/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:11.723655 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb671ef-b4c2-4255-b3b2-29c78510ab85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c81f9beca474b8fb7a5fec365cb2fca, user_id=b5edd6fd393540838f1e3a171305f647, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:11.727737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb671ef-b4c2-4255-b3b2-29c78510ab85 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:11.822442 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 872/1729] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:11 2022] PUT /identity/v3/projects/6c81f9beca474b8fb7a5fec365cb2fca/users/b5edd6fd393540838f1e3a171305f647/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:11.832674 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8593c-38b7-4a12-9751-aaec556dc6a6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:11.833195 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8593c-38b7-4a12-9751-aaec556dc6a6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:11.833472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8593c-38b7-4a12-9751-aaec556dc6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:11.915501 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3fc8593c-38b7-4a12-9751-aaec556dc6a6 None None] MFA Rules not processed for user `b5edd6fd393540838f1e3a171305f647`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:11.916864 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3fc8593c-38b7-4a12-9751-aaec556dc6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:12.037708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:12.106702 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb671ef-b4c2-4255-b3b2-29c78510ab85 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:12.108437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 858/1730] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:12.238970 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb671ef-b4c2-4255-b3b2-29c78510ab85 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:12.245513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce23d0-4208-4eae-8144-94d2011d692f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:12.246541 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce23d0-4208-4eae-8144-94d2011d692f admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:12.247006 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce23d0-4208-4eae-8144-94d2011d692f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:12.248075 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce23d0-4208-4eae-8144-94d2011d692f admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:12.249141 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce23d0-4208-4eae-8144-94d2011d692f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:12.251264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce23d0-4208-4eae-8144-94d2011d692f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:12.306601 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc8593c-38b7-4a12-9751-aaec556dc6a6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:12.313546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 873/1731] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:31:12 2022] POST /identity/v3/projects => generated 361 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:31:12.344577 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc8593c-38b7-4a12-9751-aaec556dc6a6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:12.480283 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b1872-522f-4bcf-bf7a-377480538c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:12.480932 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b1872-522f-4bcf-bf7a-377480538c1b admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:12.481213 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b1872-522f-4bcf-bf7a-377480538c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:12.481641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b1872-522f-4bcf-bf7a-377480538c1b admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:12.496037 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b1872-522f-4bcf-bf7a-377480538c1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:12.499114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b1872-522f-4bcf-bf7a-377480538c1b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:12.503423 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde734b8-3bef-4089-8b8e-be3146c7b14b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:12.504030 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde734b8-3bef-4089-8b8e-be3146c7b14b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:12.504337 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde734b8-3bef-4089-8b8e-be3146c7b14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:12.526080 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:12.568700 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 859/1732] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:31:12 2022] POST /identity/v3/users => generated 287 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:12.570671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b1872-522f-4bcf-bf7a-377480538c1b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:12.580206 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-bde734b8-3bef-4089-8b8e-be3146c7b14b None None] MFA Rules not processed for user `943fae29c5dc41f2911bbe051c74a361`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:12.584414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bde734b8-3bef-4089-8b8e-be3146c7b14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:12.587106 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b1872-522f-4bcf-bf7a-377480538c1b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:12.596843 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e92d8e-ccce-43c5-bca4-dc16382edd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:12.602150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e92d8e-ccce-43c5-bca4-dc16382edd5d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:12.602737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e92d8e-ccce-43c5-bca4-dc16382edd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:12.604084 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e92d8e-ccce-43c5-bca4-dc16382edd5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:12.606702 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e92d8e-ccce-43c5-bca4-dc16382edd5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:12.609912 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e92d8e-ccce-43c5-bca4-dc16382edd5d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:12.653126 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 860/1733] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:12.656199 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e92d8e-ccce-43c5-bca4-dc16382edd5d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:12.698740 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e92d8e-ccce-43c5-bca4-dc16382edd5d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:12.734222 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb71e9-dd3d-473c-9de7-c173a0fa5dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:12.739005 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb71e9-dd3d-473c-9de7-c173a0fa5dfa admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:12.739827 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb71e9-dd3d-473c-9de7-c173a0fa5dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:12.741274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb71e9-dd3d-473c-9de7-c173a0fa5dfa admin admin] PATH_INFO: `/v3/projects/c734b732fc3d4844b70c3035e11350e2/users/7ac21fd2138e47a5abfb7408a6d84002/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:12.755236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 874/1734] 10.222.0.70 () {58 vars in 1015 bytes} [Thu Sep 1 16:31:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:12.794618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb71e9-dd3d-473c-9de7-c173a0fa5dfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c734b732fc3d4844b70c3035e11350e2, user_id=7ac21fd2138e47a5abfb7408a6d84002, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:12.813172 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb71e9-dd3d-473c-9de7-c173a0fa5dfa admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:12.854703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 861/1735] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:12 2022] PUT /identity/v3/projects/c734b732fc3d4844b70c3035e11350e2/users/7ac21fd2138e47a5abfb7408a6d84002/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:12.855871 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde734b8-3bef-4089-8b8e-be3146c7b14b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:12.881283 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde734b8-3bef-4089-8b8e-be3146c7b14b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:12.882894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb71e9-dd3d-473c-9de7-c173a0fa5dfa admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:12.884922 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bcfb71e9-dd3d-473c-9de7-c173a0fa5dfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:13.046129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb71e9-dd3d-473c-9de7-c173a0fa5dfa admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:13.056930 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d84c38-4c11-49e5-8cd8-9925127f22c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:13.057193 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00be17-e798-4ff1-889d-46e81633d542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:13.057941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d84c38-4c11-49e5-8cd8-9925127f22c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:13.058079 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00be17-e798-4ff1-889d-46e81633d542 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:13.058387 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00be17-e798-4ff1-889d-46e81633d542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:13.058690 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00be17-e798-4ff1-889d-46e81633d542 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:13.059421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f00be17-e798-4ff1-889d-46e81633d542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:13.060913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d84c38-4c11-49e5-8cd8-9925127f22c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:13.061317 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d84c38-4c11-49e5-8cd8-9925127f22c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:13.067975 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f00be17-e798-4ff1-889d-46e81633d542 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:13.081866 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 875/1736] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:13.085465 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00be17-e798-4ff1-889d-46e81633d542 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:13.088308 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d84c38-4c11-49e5-8cd8-9925127f22c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:13.089901 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d84c38-4c11-49e5-8cd8-9925127f22c4 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:13.109800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00be17-e798-4ff1-889d-46e81633d542 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:13.111860 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 862/1737] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:31:13.123517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd20b4-81ad-46d5-82c2-683749ec0b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:13.126978 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd20b4-81ad-46d5-82c2-683749ec0b13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:13.127420 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd20b4-81ad-46d5-82c2-683749ec0b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:13.127895 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd20b4-81ad-46d5-82c2-683749ec0b13 admin admin] PATH_INFO: `/v3/projects/c734b732fc3d4844b70c3035e11350e2/users/7ac21fd2138e47a5abfb7408a6d84002/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:13.131759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd20b4-81ad-46d5-82c2-683749ec0b13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c734b732fc3d4844b70c3035e11350e2, user_id=7ac21fd2138e47a5abfb7408a6d84002, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:13.141450 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd20b4-81ad-46d5-82c2-683749ec0b13 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:13.173076 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 876/1738] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:13 2022] PUT /identity/v3/projects/c734b732fc3d4844b70c3035e11350e2/users/7ac21fd2138e47a5abfb7408a6d84002/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:13.183876 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baca9d7-ea0d-43f9-9c6f-cb39da8707fc None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:13.184239 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baca9d7-ea0d-43f9-9c6f-cb39da8707fc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:13.185155 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baca9d7-ea0d-43f9-9c6f-cb39da8707fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:13.269325 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6baca9d7-ea0d-43f9-9c6f-cb39da8707fc None None] MFA Rules not processed for user `7ac21fd2138e47a5abfb7408a6d84002`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:13.275372 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6baca9d7-ea0d-43f9-9c6f-cb39da8707fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:13.385630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:13.489743 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 863/1739] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:13 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:31:13.502113 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd20b4-81ad-46d5-82c2-683749ec0b13 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:13.508391 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-97dd20b4-81ad-46d5-82c2-683749ec0b13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:13.763541 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd20b4-81ad-46d5-82c2-683749ec0b13 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:13.771954 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2962a1-766e-454b-95d2-68e046a9c125 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:13.772709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2962a1-766e-454b-95d2-68e046a9c125 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:13.773000 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2962a1-766e-454b-95d2-68e046a9c125 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:13.773324 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2962a1-766e-454b-95d2-68e046a9c125 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:13.851349 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2962a1-766e-454b-95d2-68e046a9c125 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:13.859200 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2962a1-766e-454b-95d2-68e046a9c125 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:13.950489 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 877/1740] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:13 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:14.080015 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baca9d7-ea0d-43f9-9c6f-cb39da8707fc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:14.082917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6baca9d7-ea0d-43f9-9c6f-cb39da8707fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:14.133879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2962a1-766e-454b-95d2-68e046a9c125 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:14.144482 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2962a1-766e-454b-95d2-68e046a9c125 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:14.163136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b00c68-c00f-487c-8f3c-83d64f40ee7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:14.163600 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b00c68-c00f-487c-8f3c-83d64f40ee7d service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:14.163883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b00c68-c00f-487c-8f3c-83d64f40ee7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:14.164192 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b00c68-c00f-487c-8f3c-83d64f40ee7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:14.272242 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:14.319650 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b00c68-c00f-487c-8f3c-83d64f40ee7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:14.321124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b00c68-c00f-487c-8f3c-83d64f40ee7d service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:14.339701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 878/1741] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:14.340527 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b00c68-c00f-487c-8f3c-83d64f40ee7d service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:14.346093 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-29b00c68-c00f-487c-8f3c-83d64f40ee7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:14.412170 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baca9d7-ea0d-43f9-9c6f-cb39da8707fc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:14.440490 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc5c28-4270-4fea-b211-b6f5ed7da99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:14.441354 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc5c28-4270-4fea-b211-b6f5ed7da99c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:14.443051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc5c28-4270-4fea-b211-b6f5ed7da99c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:14.443509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc5c28-4270-4fea-b211-b6f5ed7da99c admin admin] PATH_INFO: `/v3/projects/f0d017e537c048c284664397d7ffd260` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:14.446683 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc5c28-4270-4fea-b211-b6f5ed7da99c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0d017e537c048c284664397d7ffd260)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:14.449841 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc5c28-4270-4fea-b211-b6f5ed7da99c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:14.532776 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b00c68-c00f-487c-8f3c-83d64f40ee7d service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:14.559238 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30e905-b82a-49e6-807f-d6f7f90587f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:14.560312 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30e905-b82a-49e6-807f-d6f7f90587f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:14.560875 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30e905-b82a-49e6-807f-d6f7f90587f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:14.562732 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30e905-b82a-49e6-807f-d6f7f90587f8 admin admin] PATH_INFO: `/v3/projects/746e75634f284711b7050bc02c34afc9` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:14.568171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30e905-b82a-49e6-807f-d6f7f90587f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=746e75634f284711b7050bc02c34afc9)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:14.574385 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30e905-b82a-49e6-807f-d6f7f90587f8 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:14.649961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-72cc5c28-4270-4fea-b211-b6f5ed7da99c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0d017e537c048c284664397d7ffd260', 'request_id': 'req-72cc5c28-4270-4fea-b211-b6f5ed7da99c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:14.684300 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-72cc5c28-4270-4fea-b211-b6f5ed7da99c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0d017e537c048c284664397d7ffd260', 'request_id': 'req-72cc5c28-4270-4fea-b211-b6f5ed7da99c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:14.704700 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8a30e905-b82a-49e6-807f-d6f7f90587f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '746e75634f284711b7050bc02c34afc9', 'request_id': 'req-8a30e905-b82a-49e6-807f-d6f7f90587f8'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:14.720110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8a30e905-b82a-49e6-807f-d6f7f90587f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '746e75634f284711b7050bc02c34afc9', 'request_id': 'req-8a30e905-b82a-49e6-807f-d6f7f90587f8'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:14.721201 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-72cc5c28-4270-4fea-b211-b6f5ed7da99c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0d017e537c048c284664397d7ffd260', 'request_id': 'req-72cc5c28-4270-4fea-b211-b6f5ed7da99c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:14.736471 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8a30e905-b82a-49e6-807f-d6f7f90587f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '746e75634f284711b7050bc02c34afc9', 'request_id': 'req-8a30e905-b82a-49e6-807f-d6f7f90587f8'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:14.762005 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-72cc5c28-4270-4fea-b211-b6f5ed7da99c admin admin] The token cache is being invalidate because project f0d017e537c048c284664397d7ffd260 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:31:14.762005 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-72cc5c28-4270-4fea-b211-b6f5ed7da99c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:14.768231 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-72cc5c28-4270-4fea-b211-b6f5ed7da99c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:14.773669 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8a30e905-b82a-49e6-807f-d6f7f90587f8 admin admin] The token cache is being invalidate because project 746e75634f284711b7050bc02c34afc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:31:14.773943 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8a30e905-b82a-49e6-807f-d6f7f90587f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:14.774701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8a30e905-b82a-49e6-807f-d6f7f90587f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:14.783562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 879/1742] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:31:14 2022] DELETE /identity/v3/projects/746e75634f284711b7050bc02c34afc9 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:14.787169 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 864/1743] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:31:14 2022] DELETE /identity/v3/projects/f0d017e537c048c284664397d7ffd260 => generated 0 bytes in 707 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:14.792257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92100e56-4095-41c7-9558-4a828f440490 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:14.792428 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92100e56-4095-41c7-9558-4a828f440490 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:14.792708 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92100e56-4095-41c7-9558-4a828f440490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:14.793296 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68a2f3-75f9-4383-bb91-f80a05bd9eec None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:14.793590 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68a2f3-75f9-4383-bb91-f80a05bd9eec None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:14.793912 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68a2f3-75f9-4383-bb91-f80a05bd9eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:14.827247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-92100e56-4095-41c7-9558-4a828f440490 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:14.828496 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-92100e56-4095-41c7-9558-4a828f440490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:14.924598 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-df68a2f3-75f9-4383-bb91-f80a05bd9eec None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:14.928572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-df68a2f3-75f9-4383-bb91-f80a05bd9eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:14.962888 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 880/1744] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:31:14.964086 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebcadf1-6a68-495f-bcf9-2920a877628a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:14.964375 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebcadf1-6a68-495f-bcf9-2920a877628a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:14.964650 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebcadf1-6a68-495f-bcf9-2920a877628a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:15.012312 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-3ebcadf1-6a68-495f-bcf9-2920a877628a None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:15.014083 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3ebcadf1-6a68-495f-bcf9-2920a877628a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:15.039856 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 881/1745] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:15.044136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebcadf1-6a68-495f-bcf9-2920a877628a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:15.052678 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:15.056476 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebcadf1-6a68-495f-bcf9-2920a877628a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:15.069427 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831e63e-de93-40b4-89ab-1c1856c98100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:15.069624 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:15.070056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831e63e-de93-40b4-89ab-1c1856c98100 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:15.070376 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831e63e-de93-40b4-89ab-1c1856c98100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:15.070660 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831e63e-de93-40b4-89ab-1c1856c98100 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:15.072737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831e63e-de93-40b4-89ab-1c1856c98100 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:31:15.073075 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831e63e-de93-40b4-89ab-1c1856c98100 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:15.074838 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831e63e-de93-40b4-89ab-1c1856c98100 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:15.091884 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 882/1746] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:31:15.099330 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 865/1747] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:15.109293 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831e63e-de93-40b4-89ab-1c1856c98100 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:15.109853 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cdc97-d99c-4048-9823-3d58785b3e90 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:15.110372 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cdc97-d99c-4048-9823-3d58785b3e90 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:15.110662 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cdc97-d99c-4048-9823-3d58785b3e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:15.122587 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831e63e-de93-40b4-89ab-1c1856c98100 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:15.128591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1cfc5e-df4d-46dd-97f2-ae748600baf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:15.129955 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cfc5e-df4d-46dd-97f2-ae748600baf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:15.130423 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cfc5e-df4d-46dd-97f2-ae748600baf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:15.130944 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cfc5e-df4d-46dd-97f2-ae748600baf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:15.131824 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1cfc5e-df4d-46dd-97f2-ae748600baf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:15.134089 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1cfc5e-df4d-46dd-97f2-ae748600baf6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:15.187684 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 883/1748] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:31:15 2022] POST /identity/v3/projects => generated 379 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:15.189055 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1cfc5e-df4d-46dd-97f2-ae748600baf6 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:15.193261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-cd3cdc97-d99c-4048-9823-3d58785b3e90 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:15.194582 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cd3cdc97-d99c-4048-9823-3d58785b3e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:15.270949 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1cfc5e-df4d-46dd-97f2-ae748600baf6 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:15.277176 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd3b3c-1175-43a7-89ca-bc800856cea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:15.277874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd3b3c-1175-43a7-89ca-bc800856cea5 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:15.278471 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd3b3c-1175-43a7-89ca-bc800856cea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:15.278760 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd3b3c-1175-43a7-89ca-bc800856cea5 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:15.279413 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fd3b3c-1175-43a7-89ca-bc800856cea5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:15.281658 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fd3b3c-1175-43a7-89ca-bc800856cea5 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:15.321253 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 884/1749] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:31:15 2022] POST /identity/v3/users => generated 290 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:15.322473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd3b3c-1175-43a7-89ca-bc800856cea5 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:15.348350 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd3b3c-1175-43a7-89ca-bc800856cea5 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:15.359032 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e25ba-cff3-4134-a10b-36d7cd2871ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:15.360422 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e25ba-cff3-4134-a10b-36d7cd2871ff admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:15.362074 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e25ba-cff3-4134-a10b-36d7cd2871ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:15.362556 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e25ba-cff3-4134-a10b-36d7cd2871ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:15.364264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e25ba-cff3-4134-a10b-36d7cd2871ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:15.368416 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e25ba-cff3-4134-a10b-36d7cd2871ff admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:15.409170 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 866/1750] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:15.410699 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3cdc97-d99c-4048-9823-3d58785b3e90 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:15.414815 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 885/1751] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:15.419373 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e25ba-cff3-4134-a10b-36d7cd2871ff admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:15.426033 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3cdc97-d99c-4048-9823-3d58785b3e90 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:15.441844 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7f72c-e4e1-4c51-b5cd-e2d7c62359a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:15.443444 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7f72c-e4e1-4c51-b5cd-e2d7c62359a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:15.444019 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7f72c-e4e1-4c51-b5cd-e2d7c62359a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:15.444484 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7f72c-e4e1-4c51-b5cd-e2d7c62359a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:15.445395 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7f72c-e4e1-4c51-b5cd-e2d7c62359a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:31:15.445575 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e25ba-cff3-4134-a10b-36d7cd2871ff admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:15.446057 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7f72c-e4e1-4c51-b5cd-e2d7c62359a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:15.448131 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7f72c-e4e1-4c51-b5cd-e2d7c62359a8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:15.466439 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3242f-a5f4-4697-b059-fed6b4e45211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:15.467224 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3242f-a5f4-4697-b059-fed6b4e45211 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:15.471758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3242f-a5f4-4697-b059-fed6b4e45211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:15.472202 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3242f-a5f4-4697-b059-fed6b4e45211 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:15.472995 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea3242f-a5f4-4697-b059-fed6b4e45211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:15.479013 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea3242f-a5f4-4697-b059-fed6b4e45211 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:15.494169 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 867/1752] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:31:15.496088 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7f72c-e4e1-4c51-b5cd-e2d7c62359a8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:15.516962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7f72c-e4e1-4c51-b5cd-e2d7c62359a8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:15.524699 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 886/1753] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:15.530921 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3242f-a5f4-4697-b059-fed6b4e45211 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:15.535071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f12aa0-28cd-49a6-8025-52112cc49b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:15.535918 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f12aa0-28cd-49a6-8025-52112cc49b4c admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:15.537352 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f12aa0-28cd-49a6-8025-52112cc49b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:15.538769 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f12aa0-28cd-49a6-8025-52112cc49b4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:15.539743 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f12aa0-28cd-49a6-8025-52112cc49b4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:15.541851 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f12aa0-28cd-49a6-8025-52112cc49b4c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:15.574772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3242f-a5f4-4697-b059-fed6b4e45211 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:15.604739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8a279-f661-442e-a81e-d53a31d26997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:15.608926 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 868/1754] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:31:15 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:15.613109 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8a279-f661-442e-a81e-d53a31d26997 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:15.613109 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8a279-f661-442e-a81e-d53a31d26997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:15.613109 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8a279-f661-442e-a81e-d53a31d26997 admin admin] PATH_INFO: `/v3/projects/57c23332fe6e467d9200920e4d0fe593/users/e453798028f84a92af8b34238c129065/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:15.617665 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f12aa0-28cd-49a6-8025-52112cc49b4c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:15.688700 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e8a279-f661-442e-a81e-d53a31d26997 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57c23332fe6e467d9200920e4d0fe593, user_id=e453798028f84a92af8b34238c129065, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:15.692567 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e8a279-f661-442e-a81e-d53a31d26997 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:15.736637 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 887/1755] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:15 2022] PUT /identity/v3/projects/57c23332fe6e467d9200920e4d0fe593/users/e453798028f84a92af8b34238c129065/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:15.738207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ef2f4-ac80-4c00-8c88-c61bfae2b199 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:15.738526 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ef2f4-ac80-4c00-8c88-c61bfae2b199 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:15.741806 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ef2f4-ac80-4c00-8c88-c61bfae2b199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:15.793627 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-919ef2f4-ac80-4c00-8c88-c61bfae2b199 None None] MFA Rules not processed for user `4ecfc51400414558b8389a9c35b2fa1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:15.795346 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-919ef2f4-ac80-4c00-8c88-c61bfae2b199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:15.886831 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f12aa0-28cd-49a6-8025-52112cc49b4c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:15.976899 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 888/1756] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:15.980388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff10de0-39f4-457c-ada0-cc5d0fbb9561 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:15.980681 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff10de0-39f4-457c-ada0-cc5d0fbb9561 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:15.980990 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff10de0-39f4-457c-ada0-cc5d0fbb9561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:16.115244 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-aff10de0-39f4-457c-ada0-cc5d0fbb9561 None None] MFA Rules not processed for user `e453798028f84a92af8b34238c129065`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:16.117350 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-aff10de0-39f4-457c-ada0-cc5d0fbb9561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:16.135284 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ebfd4-eead-4419-9459-d471eacca17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:16.137266 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ebfd4-eead-4419-9459-d471eacca17d admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:16.137890 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:16.138119 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ebfd4-eead-4419-9459-d471eacca17d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:16.145506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ebfd4-eead-4419-9459-d471eacca17d admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:16.145506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ebfd4-eead-4419-9459-d471eacca17d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:16.145506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ebfd4-eead-4419-9459-d471eacca17d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:16.261659 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 869/1757] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:31:15 2022] POST /identity/v3/users => generated 294 bytes in 647 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:16.264774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ebfd4-eead-4419-9459-d471eacca17d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:16.296249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:16.323248 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ebfd4-eead-4419-9459-d471eacca17d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:16.346681 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13559976-3efc-4583-9379-b313e7cab4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:16.347670 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13559976-3efc-4583-9379-b313e7cab4e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:16.348122 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13559976-3efc-4583-9379-b313e7cab4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:16.348588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13559976-3efc-4583-9379-b313e7cab4e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:16.349463 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13559976-3efc-4583-9379-b313e7cab4e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:16.351311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13559976-3efc-4583-9379-b313e7cab4e9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:16.376751 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 870/1758] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:16.382140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13559976-3efc-4583-9379-b313e7cab4e9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:16.395632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13559976-3efc-4583-9379-b313e7cab4e9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:16.415585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c384368-62ef-4fbd-9880-7ac6914dfc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:16.416544 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c384368-62ef-4fbd-9880-7ac6914dfc0d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:16.417110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c384368-62ef-4fbd-9880-7ac6914dfc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:16.418922 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c384368-62ef-4fbd-9880-7ac6914dfc0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:16.419416 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 889/1759] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 442 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:16.420293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c384368-62ef-4fbd-9880-7ac6914dfc0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:16.423031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c384368-62ef-4fbd-9880-7ac6914dfc0d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:16.424132 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff10de0-39f4-457c-ada0-cc5d0fbb9561 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:16.453202 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 871/1760] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:16.466529 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff10de0-39f4-457c-ada0-cc5d0fbb9561 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:16.472193 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c384368-62ef-4fbd-9880-7ac6914dfc0d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:16.496034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2fe17b-874f-44f2-b74e-6134720c3dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:16.496765 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fe17b-874f-44f2-b74e-6134720c3dc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:16.497202 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fe17b-874f-44f2-b74e-6134720c3dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:16.497590 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fe17b-874f-44f2-b74e-6134720c3dc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:16.498377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2fe17b-874f-44f2-b74e-6134720c3dc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:16.501436 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2fe17b-874f-44f2-b74e-6134720c3dc7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:16.513984 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c384368-62ef-4fbd-9880-7ac6914dfc0d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:16.527715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1fb56a-2dfb-4653-9570-5b9ae1662fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:16.529999 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fb56a-2dfb-4653-9570-5b9ae1662fd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:16.531605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fb56a-2dfb-4653-9570-5b9ae1662fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:16.531897 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fb56a-2dfb-4653-9570-5b9ae1662fd9 admin admin] PATH_INFO: `/v3/projects/80d47cc2ac594110b9187f6cfbdfbff6/users/8ac259f9fa364c639025178e7e7647c6/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:16.641911 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1fb56a-2dfb-4653-9570-5b9ae1662fd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80d47cc2ac594110b9187f6cfbdfbff6, user_id=8ac259f9fa364c639025178e7e7647c6, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:16.647135 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 890/1761] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:31:16 2022] POST /identity/v3/projects => generated 377 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:31:16.651625 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2fe17b-874f-44f2-b74e-6134720c3dc7 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:16.655472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1fb56a-2dfb-4653-9570-5b9ae1662fd9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:16.766414 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 872/1762] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:16 2022] PUT /identity/v3/projects/80d47cc2ac594110b9187f6cfbdfbff6/users/8ac259f9fa364c639025178e7e7647c6/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:16.777928 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc369f6-2a25-4aa7-b17b-6f72d32427c4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:16.778374 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc369f6-2a25-4aa7-b17b-6f72d32427c4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:16.779287 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc369f6-2a25-4aa7-b17b-6f72d32427c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:16.873550 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8fc369f6-2a25-4aa7-b17b-6f72d32427c4 None None] MFA Rules not processed for user `8ac259f9fa364c639025178e7e7647c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:16.875044 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8fc369f6-2a25-4aa7-b17b-6f72d32427c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:16.889814 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2fe17b-874f-44f2-b74e-6134720c3dc7 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:17.038783 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d93af-ded6-4808-ab5e-28461e241c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:17.039307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d93af-ded6-4808-ab5e-28461e241c4f admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:17.039597 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d93af-ded6-4808-ab5e-28461e241c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:17.039874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d93af-ded6-4808-ab5e-28461e241c4f admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:17.040611 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d93af-ded6-4808-ab5e-28461e241c4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:17.042583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d93af-ded6-4808-ab5e-28461e241c4f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:17.123331 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 891/1763] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:31:16 2022] POST /identity/v3/users => generated 289 bytes in 474 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:17.128408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d93af-ded6-4808-ab5e-28461e241c4f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:17.144169 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d93af-ded6-4808-ab5e-28461e241c4f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:17.174385 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c899d8-b769-4d2a-815a-9a3afa7837ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:17.177594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c899d8-b769-4d2a-815a-9a3afa7837ef admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:17.178963 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c899d8-b769-4d2a-815a-9a3afa7837ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:17.179426 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c899d8-b769-4d2a-815a-9a3afa7837ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:17.181683 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c899d8-b769-4d2a-815a-9a3afa7837ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:17.199983 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c899d8-b769-4d2a-815a-9a3afa7837ef admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:17.234707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 892/1764] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:17.238028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c899d8-b769-4d2a-815a-9a3afa7837ef admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:17.239332 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 873/1765] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:17.240447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc369f6-2a25-4aa7-b17b-6f72d32427c4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:17.286022 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc369f6-2a25-4aa7-b17b-6f72d32427c4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:17.296990 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c899d8-b769-4d2a-815a-9a3afa7837ef admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:17.311911 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598bbef-c48a-4bfa-8ce4-b3070e945695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:17.312544 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598bbef-c48a-4bfa-8ce4-b3070e945695 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:17.313491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598bbef-c48a-4bfa-8ce4-b3070e945695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:17.313782 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598bbef-c48a-4bfa-8ce4-b3070e945695 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:17.314589 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598bbef-c48a-4bfa-8ce4-b3070e945695 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:17.316529 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598bbef-c48a-4bfa-8ce4-b3070e945695 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:17.330034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306bca1-5312-4acd-a963-269cc357864a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:17.330691 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306bca1-5312-4acd-a963-269cc357864a admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:17.330972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306bca1-5312-4acd-a963-269cc357864a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:17.331243 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306bca1-5312-4acd-a963-269cc357864a admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:17.331897 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306bca1-5312-4acd-a963-269cc357864a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:17.333532 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306bca1-5312-4acd-a963-269cc357864a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:17.343687 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:17.396096 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 893/1766] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:17.407480 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306bca1-5312-4acd-a963-269cc357864a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:17.445270 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 874/1767] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:31:17 2022] POST /identity/v3/projects => generated 387 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:17.446423 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306bca1-5312-4acd-a963-269cc357864a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:17.449191 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598bbef-c48a-4bfa-8ce4-b3070e945695 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:17.654095 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598bbef-c48a-4bfa-8ce4-b3070e945695 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:17.666410 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbaf3ef-9f2a-4566-ad31-746659f21539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:17.668379 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbaf3ef-9f2a-4566-ad31-746659f21539 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:17.670519 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbaf3ef-9f2a-4566-ad31-746659f21539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:17.671847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbaf3ef-9f2a-4566-ad31-746659f21539 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:17.673547 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbaf3ef-9f2a-4566-ad31-746659f21539 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:17.680232 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbaf3ef-9f2a-4566-ad31-746659f21539 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:17.701164 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a43ce5-0cca-452f-ace3-2c34a7a6e67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:17.702452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a43ce5-0cca-452f-ace3-2c34a7a6e67d admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:17.702900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a43ce5-0cca-452f-ace3-2c34a7a6e67d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:17.703830 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a43ce5-0cca-452f-ace3-2c34a7a6e67d admin admin] PATH_INFO: `/v3/projects/96dcecfa7b9b4c28b736860a392029fb/users/b48ae6ee306d4f2b9539c5c0b2f29cc2/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:17.750061 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a43ce5-0cca-452f-ace3-2c34a7a6e67d admin admin] RBAC: Authorizing `identity:create_grant(project_id=96dcecfa7b9b4c28b736860a392029fb, user_id=b48ae6ee306d4f2b9539c5c0b2f29cc2, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:17.751420 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 875/1768] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:31:17 2022] POST /identity/v3/users => generated 300 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:17.755168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbaf3ef-9f2a-4566-ad31-746659f21539 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:17.760164 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a43ce5-0cca-452f-ace3-2c34a7a6e67d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:17.784198 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbaf3ef-9f2a-4566-ad31-746659f21539 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:17.808494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:17.810762 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 894/1769] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:17 2022] PUT /identity/v3/projects/96dcecfa7b9b4c28b736860a392029fb/users/b48ae6ee306d4f2b9539c5c0b2f29cc2/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:17.812934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c3eef-d4b8-4f3d-aab4-0fc5129bf657 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:17.813290 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c3eef-d4b8-4f3d-aab4-0fc5129bf657 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:17.813680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c3eef-d4b8-4f3d-aab4-0fc5129bf657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:17.858016 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-1b6c3eef-d4b8-4f3d-aab4-0fc5129bf657 None None] MFA Rules not processed for user `b48ae6ee306d4f2b9539c5c0b2f29cc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:17.859542 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1b6c3eef-d4b8-4f3d-aab4-0fc5129bf657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:17.989807 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 895/1770] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:17.996001 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c3eef-d4b8-4f3d-aab4-0fc5129bf657 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:18.004645 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a588998-a382-496b-a4a7-99f8f2ddf5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:18.008628 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a588998-a382-496b-a4a7-99f8f2ddf5bd admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:18.008628 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a588998-a382-496b-a4a7-99f8f2ddf5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:18.008628 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a588998-a382-496b-a4a7-99f8f2ddf5bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:18.008628 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a588998-a382-496b-a4a7-99f8f2ddf5bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:18.008628 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a588998-a382-496b-a4a7-99f8f2ddf5bd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:18.032598 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c3eef-d4b8-4f3d-aab4-0fc5129bf657 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:18.048645 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30aced-63f6-4ca1-9957-bbac64f69ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:18.049279 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30aced-63f6-4ca1-9957-bbac64f69ac9 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:18.049615 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30aced-63f6-4ca1-9957-bbac64f69ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:18.049893 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30aced-63f6-4ca1-9957-bbac64f69ac9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:18.050930 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de30aced-63f6-4ca1-9957-bbac64f69ac9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:18.052998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de30aced-63f6-4ca1-9957-bbac64f69ac9 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:18.062988 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 876/1771] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:18.066595 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a588998-a382-496b-a4a7-99f8f2ddf5bd admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:18.096791 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a588998-a382-496b-a4a7-99f8f2ddf5bd admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:18.130549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7c770-b40b-4009-bae9-185563b56840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:18.135117 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7c770-b40b-4009-bae9-185563b56840 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:18.135556 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7c770-b40b-4009-bae9-185563b56840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:18.141690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7c770-b40b-4009-bae9-185563b56840 admin admin] PATH_INFO: `/v3/projects/4ee5ddb8080648edaae0f0ff46da4d21/users/afa277d8e76448e693af064219fe484d/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:18.191086 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 896/1772] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:31:17 2022] POST /identity/v3/projects => generated 377 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:18.192040 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30aced-63f6-4ca1-9957-bbac64f69ac9 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:18.219418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a7c770-b40b-4009-bae9-185563b56840 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ee5ddb8080648edaae0f0ff46da4d21, user_id=afa277d8e76448e693af064219fe484d, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:18.223346 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a7c770-b40b-4009-bae9-185563b56840 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:18.271303 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 877/1773] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:18 2022] PUT /identity/v3/projects/4ee5ddb8080648edaae0f0ff46da4d21/users/afa277d8e76448e693af064219fe484d/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:18.275319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7c770-b40b-4009-bae9-185563b56840 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:18.397037 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:18.477748 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30aced-63f6-4ca1-9957-bbac64f69ac9 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:18.535304 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7c770-b40b-4009-bae9-185563b56840 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:18.554481 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530f802-2f04-40a6-a8c1-43adc2becbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:18.556684 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530f802-2f04-40a6-a8c1-43adc2becbff admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:18.557016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530f802-2f04-40a6-a8c1-43adc2becbff admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:18.557303 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530f802-2f04-40a6-a8c1-43adc2becbff admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:18.557973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d530f802-2f04-40a6-a8c1-43adc2becbff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:18.564150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d530f802-2f04-40a6-a8c1-43adc2becbff admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:18.608379 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 878/1774] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:18.610149 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530f802-2f04-40a6-a8c1-43adc2becbff admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:18.661661 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530f802-2f04-40a6-a8c1-43adc2becbff admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:18.683551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9530dc5-2408-4f27-b129-1a847e413dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:18.689733 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9530dc5-2408-4f27-b129-1a847e413dd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:18.690758 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9530dc5-2408-4f27-b129-1a847e413dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:18.691197 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9530dc5-2408-4f27-b129-1a847e413dd5 admin admin] PATH_INFO: `/v3/projects/4ee5ddb8080648edaae0f0ff46da4d21/users/afa277d8e76448e693af064219fe484d/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:18.703757 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b48548-1dc7-44e3-886d-1b584b2c2ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:18.710396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b48548-1dc7-44e3-886d-1b584b2c2ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:18.710704 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b48548-1dc7-44e3-886d-1b584b2c2ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:18.710997 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b48548-1dc7-44e3-886d-1b584b2c2ed5 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:18.717306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9530dc5-2408-4f27-b129-1a847e413dd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ee5ddb8080648edaae0f0ff46da4d21, user_id=afa277d8e76448e693af064219fe484d, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:18.721358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b48548-1dc7-44e3-886d-1b584b2c2ed5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:18.726802 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b48548-1dc7-44e3-886d-1b584b2c2ed5 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:18.736304 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9530dc5-2408-4f27-b129-1a847e413dd5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:18.765909 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 879/1775] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:18 2022] PUT /identity/v3/projects/4ee5ddb8080648edaae0f0ff46da4d21/users/afa277d8e76448e693af064219fe484d/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:31:18.767231 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9530dc5-2408-4f27-b129-1a847e413dd5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:18.778939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f9530dc5-2408-4f27-b129-1a847e413dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:18.836409 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 897/1776] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:31:18 2022] POST /identity/v3/users => generated 295 bytes in 645 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:18.839954 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f386e4-a102-4e2f-a4ee-03ee84d2abcc None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:18.841422 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f386e4-a102-4e2f-a4ee-03ee84d2abcc None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:18.842890 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f386e4-a102-4e2f-a4ee-03ee84d2abcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:18.958362 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-44f386e4-a102-4e2f-a4ee-03ee84d2abcc None None] MFA Rules not processed for user `afa277d8e76448e693af064219fe484d`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:18.960393 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-44f386e4-a102-4e2f-a4ee-03ee84d2abcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:19.014771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9530dc5-2408-4f27-b129-1a847e413dd5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:19.051412 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e18ec0-9b40-48cb-9499-68b1e3a3f3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:19.052093 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e18ec0-9b40-48cb-9499-68b1e3a3f3e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:19.052336 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e18ec0-9b40-48cb-9499-68b1e3a3f3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:19.052602 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e18ec0-9b40-48cb-9499-68b1e3a3f3e3 admin admin] PATH_INFO: `/v3/users/b5edd6fd393540838f1e3a171305f647` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:19.057870 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e18ec0-9b40-48cb-9499-68b1e3a3f3e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5edd6fd393540838f1e3a171305f647)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:19.068360 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e18ec0-9b40-48cb-9499-68b1e3a3f3e3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:19.088873 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:19.222882 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 898/1777] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:18 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:19.225482 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f386e4-a102-4e2f-a4ee-03ee84d2abcc None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:19.241187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3e18ec0-9b40-48cb-9499-68b1e3a3f3e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5edd6fd393540838f1e3a171305f647'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:19.254229 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3e18ec0-9b40-48cb-9499-68b1e3a3f3e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5edd6fd393540838f1e3a171305f647'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:19.255842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3e18ec0-9b40-48cb-9499-68b1e3a3f3e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5edd6fd393540838f1e3a171305f647'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:19.277027 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3e18ec0-9b40-48cb-9499-68b1e3a3f3e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5edd6fd393540838f1e3a171305f647'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:19.288955 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f386e4-a102-4e2f-a4ee-03ee84d2abcc None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:19.301116 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9531adf-0d6b-44ea-840a-53ba5d5fb63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:19.301803 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9531adf-0d6b-44ea-840a-53ba5d5fb63f admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:19.302090 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9531adf-0d6b-44ea-840a-53ba5d5fb63f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:19.303534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9531adf-0d6b-44ea-840a-53ba5d5fb63f admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:19.304239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9531adf-0d6b-44ea-840a-53ba5d5fb63f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:19.308710 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3e18ec0-9b40-48cb-9499-68b1e3a3f3e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5edd6fd393540838f1e3a171305f647'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:19.310147 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9531adf-0d6b-44ea-840a-53ba5d5fb63f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:19.325013 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 880/1778] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:31:18 2022] DELETE /identity/v3/users/b5edd6fd393540838f1e3a171305f647 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:19.326059 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e18ec0-9b40-48cb-9499-68b1e3a3f3e3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:19.330093 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a3e18ec0-9b40-48cb-9499-68b1e3a3f3e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:19.335634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 899/1779] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:19 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:19.336888 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff65363-0e65-4c3a-b296-73c6b2e6eed3 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:19.338125 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff65363-0e65-4c3a-b296-73c6b2e6eed3 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:19.338277 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff65363-0e65-4c3a-b296-73c6b2e6eed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:19.390734 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-1ff65363-0e65-4c3a-b296-73c6b2e6eed3 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:19.394850 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1ff65363-0e65-4c3a-b296-73c6b2e6eed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:19.489679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e18ec0-9b40-48cb-9499-68b1e3a3f3e3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:19.497306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56754ba8-83f0-4c7c-bdd8-d115e738e9f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:19.498321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56754ba8-83f0-4c7c-bdd8-d115e738e9f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:19.498549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56754ba8-83f0-4c7c-bdd8-d115e738e9f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:19.498840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56754ba8-83f0-4c7c-bdd8-d115e738e9f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:19.500876 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-56754ba8-83f0-4c7c-bdd8-d115e738e9f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:19.616583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:19.674340 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 900/1780] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:19.675985 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff65363-0e65-4c3a-b296-73c6b2e6eed3 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:19.684021 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1ff65363-0e65-4c3a-b296-73c6b2e6eed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:19.692970 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56754ba8-83f0-4c7c-bdd8-d115e738e9f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:19.700433 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56754ba8-83f0-4c7c-bdd8-d115e738e9f4 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:19.710430 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff65363-0e65-4c3a-b296-73c6b2e6eed3 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:19.740777 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4c244-205e-4432-aa52-c9a996069d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:19.742878 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4c244-205e-4432-aa52-c9a996069d07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:19.743113 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 881/1781] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:19 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:19.743940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4c244-205e-4432-aa52-c9a996069d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:19.745525 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4c244-205e-4432-aa52-c9a996069d07 admin admin] PATH_INFO: `/v3/projects/eb5920d73c3a447181bc8ab452dc8924/users/32701be1bd624de6afe819d743d56c99/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:19.769156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be4c244-205e-4432-aa52-c9a996069d07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb5920d73c3a447181bc8ab452dc8924, user_id=32701be1bd624de6afe819d743d56c99, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:19.772676 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be4c244-205e-4432-aa52-c9a996069d07 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:19.809315 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55474472-f546-4a1a-ab8e-218108a2d4be None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:19.812702 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55474472-f546-4a1a-ab8e-218108a2d4be None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:19.813192 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55474472-f546-4a1a-ab8e-218108a2d4be None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:19.815812 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 882/1782] 10.222.0.58 () {54 vars in 921 bytes} [Thu Sep 1 16:31:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:31:19.827204 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55474472-f546-4a1a-ab8e-218108a2d4be None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:19.829163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-55474472-f546-4a1a-ab8e-218108a2d4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:19.868992 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 901/1783] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:19 2022] PUT /identity/v3/projects/eb5920d73c3a447181bc8ab452dc8924/users/32701be1bd624de6afe819d743d56c99/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:19.883588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af25b4-f1d5-4f6f-b44f-e07774398ecb None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:19.884098 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af25b4-f1d5-4f6f-b44f-e07774398ecb None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:19.884569 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af25b4-f1d5-4f6f-b44f-e07774398ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:19.947223 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-24af25b4-f1d5-4f6f-b44f-e07774398ecb None None] MFA Rules not processed for user `1f0c99fdf8bd4331ad7e641ee38554c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:19.951918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-24af25b4-f1d5-4f6f-b44f-e07774398ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:19.957311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:19.995195 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55474472-f546-4a1a-ab8e-218108a2d4be None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:20.006493 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41e91c-35e2-4795-a222-6941fed1c0b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:20.007133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41e91c-35e2-4795-a222-6941fed1c0b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:20.007431 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41e91c-35e2-4795-a222-6941fed1c0b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:20.007702 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41e91c-35e2-4795-a222-6941fed1c0b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:20.163861 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba41e91c-35e2-4795-a222-6941fed1c0b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:20.165588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba41e91c-35e2-4795-a222-6941fed1c0b2 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:20.186413 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 902/1784] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Sep 1 16:31:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:20.188811 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af25b4-f1d5-4f6f-b44f-e07774398ecb None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:20.191580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 883/1785] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:31:20.193645 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41e91c-35e2-4795-a222-6941fed1c0b2 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:20.198461 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ba41e91c-35e2-4795-a222-6941fed1c0b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:20.211225 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af25b4-f1d5-4f6f-b44f-e07774398ecb None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:20.236790 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20212c18-818f-4c7a-89b5-dbfaacd5a70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:20.237901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20212c18-818f-4c7a-89b5-dbfaacd5a70d admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:20.238500 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20212c18-818f-4c7a-89b5-dbfaacd5a70d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:20.238855 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20212c18-818f-4c7a-89b5-dbfaacd5a70d admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:20.239699 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20212c18-818f-4c7a-89b5-dbfaacd5a70d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:20.242733 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20212c18-818f-4c7a-89b5-dbfaacd5a70d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:20.244888 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41e91c-35e2-4795-a222-6941fed1c0b2 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:20.259595 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b246b45-8c82-4f3b-be52-233e959d9de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:20.260408 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b246b45-8c82-4f3b-be52-233e959d9de4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:20.261026 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b246b45-8c82-4f3b-be52-233e959d9de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:20.261380 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b246b45-8c82-4f3b-be52-233e959d9de4 admin admin] PATH_INFO: `/v3/users/8ac259f9fa364c639025178e7e7647c6` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:20.263692 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b246b45-8c82-4f3b-be52-233e959d9de4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ac259f9fa364c639025178e7e7647c6)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:20.267544 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b246b45-8c82-4f3b-be52-233e959d9de4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:20.277830 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 903/1786] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:20.284333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20212c18-818f-4c7a-89b5-dbfaacd5a70d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:20.313955 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20212c18-818f-4c7a-89b5-dbfaacd5a70d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:20.324607 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d19bd5-2d52-46fd-8111-ee547ab46170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:20.326010 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d19bd5-2d52-46fd-8111-ee547ab46170 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:20.326392 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d19bd5-2d52-46fd-8111-ee547ab46170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:20.326742 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d19bd5-2d52-46fd-8111-ee547ab46170 admin admin] PATH_INFO: `/v3/projects/eb5920d73c3a447181bc8ab452dc8924/users/32701be1bd624de6afe819d743d56c99/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:20.332208 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d19bd5-2d52-46fd-8111-ee547ab46170 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb5920d73c3a447181bc8ab452dc8924, user_id=32701be1bd624de6afe819d743d56c99, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:20.338592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d19bd5-2d52-46fd-8111-ee547ab46170 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:20.361356 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5b246b45-8c82-4f3b-be52-233e959d9de4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ac259f9fa364c639025178e7e7647c6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:20.377922 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 904/1787] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:20 2022] PUT /identity/v3/projects/eb5920d73c3a447181bc8ab452dc8924/users/32701be1bd624de6afe819d743d56c99/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:20.380455 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e140363-9fe3-4422-86e5-f8be45c6cff9 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:20.382101 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e140363-9fe3-4422-86e5-f8be45c6cff9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:20.383119 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5b246b45-8c82-4f3b-be52-233e959d9de4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ac259f9fa364c639025178e7e7647c6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:20.383260 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e140363-9fe3-4422-86e5-f8be45c6cff9 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:20.386583 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5b246b45-8c82-4f3b-be52-233e959d9de4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ac259f9fa364c639025178e7e7647c6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:20.388380 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 905/1788] 10.222.0.58 () {54 vars in 921 bytes} [Thu Sep 1 16:31:20 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:31:20.390375 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baedf76-3624-4b71-ad02-1671b43d5f57 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:20.391391 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baedf76-3624-4b71-ad02-1671b43d5f57 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:20.391814 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baedf76-3624-4b71-ad02-1671b43d5f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:20.407372 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5b246b45-8c82-4f3b-be52-233e959d9de4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ac259f9fa364c639025178e7e7647c6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:20.448118 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5b246b45-8c82-4f3b-be52-233e959d9de4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ac259f9fa364c639025178e7e7647c6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:20.469530 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 884/1789] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:31:20 2022] DELETE /identity/v3/users/8ac259f9fa364c639025178e7e7647c6 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:20.479263 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-6baedf76-3624-4b71-ad02-1671b43d5f57 None None] MFA Rules not processed for user `32701be1bd624de6afe819d743d56c99`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:20.484549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42994ed1-7a85-454a-a418-73b8b07a1262 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:20.484843 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42994ed1-7a85-454a-a418-73b8b07a1262 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:20.485101 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6baedf76-3624-4b71-ad02-1671b43d5f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:20.485215 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42994ed1-7a85-454a-a418-73b8b07a1262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:20.579163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-42994ed1-7a85-454a-a418-73b8b07a1262 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:20.584725 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-42994ed1-7a85-454a-a418-73b8b07a1262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:20.670720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 906/1790] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:20 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:20.672184 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a554bc4-3659-4b5c-9a23-94b1668cf5ad None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:20.674180 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a554bc4-3659-4b5c-9a23-94b1668cf5ad None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:20.674662 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a554bc4-3659-4b5c-9a23-94b1668cf5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:20.725831 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-4a554bc4-3659-4b5c-9a23-94b1668cf5ad None None] MFA Rules not processed for user `1f0c99fdf8bd4331ad7e641ee38554c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:20.729099 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4a554bc4-3659-4b5c-9a23-94b1668cf5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:20.815878 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:20.851402 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:20.882270 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 885/1791] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:20.886290 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42994ed1-7a85-454a-a418-73b8b07a1262 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:20.891141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-42994ed1-7a85-454a-a418-73b8b07a1262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:20.970769 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 907/1792] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Sep 1 16:31:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:21.132005 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a554bc4-3659-4b5c-9a23-94b1668cf5ad None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:21.133516 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4a554bc4-3659-4b5c-9a23-94b1668cf5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:21.199781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42994ed1-7a85-454a-a418-73b8b07a1262 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:21.214857 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd182a-2e2c-4c63-8a26-1211baa64830 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:21.216836 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd182a-2e2c-4c63-8a26-1211baa64830 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:21.217675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd182a-2e2c-4c63-8a26-1211baa64830 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:21.218481 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd182a-2e2c-4c63-8a26-1211baa64830 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:21.250622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd182a-2e2c-4c63-8a26-1211baa64830 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:21.260080 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd182a-2e2c-4c63-8a26-1211baa64830 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:21.330530 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 886/1793] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:20 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:21.413702 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a554bc4-3659-4b5c-9a23-94b1668cf5ad None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:21.419476 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd182a-2e2c-4c63-8a26-1211baa64830 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:21.427460 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e539361-400e-42c5-9476-87e5b9bdbb1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:21.427460 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e539361-400e-42c5-9476-87e5b9bdbb1d service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:21.427460 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e539361-400e-42c5-9476-87e5b9bdbb1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:21.427460 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e539361-400e-42c5-9476-87e5b9bdbb1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:21.428131 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b9cd182a-2e2c-4c63-8a26-1211baa64830 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:21.447909 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd182a-2e2c-4c63-8a26-1211baa64830 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:21.473958 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e539361-400e-42c5-9476-87e5b9bdbb1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:21.479392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a952a-c09e-4940-ae85-bd9e6226ed5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:21.479634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e539361-400e-42c5-9476-87e5b9bdbb1d service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:21.480036 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a952a-c09e-4940-ae85-bd9e6226ed5a service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:21.480319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a952a-c09e-4940-ae85-bd9e6226ed5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:21.480598 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a952a-c09e-4940-ae85-bd9e6226ed5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:21.487373 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-660a952a-c09e-4940-ae85-bd9e6226ed5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:21.517899 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 908/1794] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:21.693041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a952a-c09e-4940-ae85-bd9e6226ed5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:21.698384 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a952a-c09e-4940-ae85-bd9e6226ed5a service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:21.730119 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 887/1795] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:22.997260 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e539361-400e-42c5-9476-87e5b9bdbb1d service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:22.999452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1e539361-400e-42c5-9476-87e5b9bdbb1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:23.026560 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e539361-400e-42c5-9476-87e5b9bdbb1d service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:23.035770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1e76b-7d32-4190-8b8e-c21da37fd3a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:23.036707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1e76b-7d32-4190-8b8e-c21da37fd3a9 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:23.037044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1e76b-7d32-4190-8b8e-c21da37fd3a9 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:23.037421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1e76b-7d32-4190-8b8e-c21da37fd3a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:23.039913 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ebb1e76b-7d32-4190-8b8e-c21da37fd3a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:23.073438 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb1e76b-7d32-4190-8b8e-c21da37fd3a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:23.077577 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb1e76b-7d32-4190-8b8e-c21da37fd3a9 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:23.121976 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 909/1796] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:23.230518 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a952a-c09e-4940-ae85-bd9e6226ed5a service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:23.232148 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-660a952a-c09e-4940-ae85-bd9e6226ed5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:23.266902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a952a-c09e-4940-ae85-bd9e6226ed5a service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:23.274068 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e74fb9-568a-4cf5-8c6d-6918ae562c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:23.274968 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e74fb9-568a-4cf5-8c6d-6918ae562c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:23.275312 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e74fb9-568a-4cf5-8c6d-6918ae562c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:23.275606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e74fb9-568a-4cf5-8c6d-6918ae562c78 admin admin] PATH_INFO: `/v3/projects/6c81f9beca474b8fb7a5fec365cb2fca` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:23.278360 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e74fb9-568a-4cf5-8c6d-6918ae562c78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c81f9beca474b8fb7a5fec365cb2fca)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:23.280518 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e74fb9-568a-4cf5-8c6d-6918ae562c78 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:23.391042 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-84e74fb9-568a-4cf5-8c6d-6918ae562c78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c81f9beca474b8fb7a5fec365cb2fca', 'request_id': 'req-84e74fb9-568a-4cf5-8c6d-6918ae562c78'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:23.400029 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-84e74fb9-568a-4cf5-8c6d-6918ae562c78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c81f9beca474b8fb7a5fec365cb2fca', 'request_id': 'req-84e74fb9-568a-4cf5-8c6d-6918ae562c78'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:23.414362 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1e76b-7d32-4190-8b8e-c21da37fd3a9 service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:23.415986 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ebb1e76b-7d32-4190-8b8e-c21da37fd3a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:23.442271 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1e76b-7d32-4190-8b8e-c21da37fd3a9 service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:23.447444 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-84e74fb9-568a-4cf5-8c6d-6918ae562c78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c81f9beca474b8fb7a5fec365cb2fca', 'request_id': 'req-84e74fb9-568a-4cf5-8c6d-6918ae562c78'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:23.479791 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-84e74fb9-568a-4cf5-8c6d-6918ae562c78 admin admin] The token cache is being invalidate because project 6c81f9beca474b8fb7a5fec365cb2fca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:31:23.480334 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-84e74fb9-568a-4cf5-8c6d-6918ae562c78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:23.481391 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-84e74fb9-568a-4cf5-8c6d-6918ae562c78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:23.484386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 888/1797] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:31:23 2022] DELETE /identity/v3/projects/6c81f9beca474b8fb7a5fec365cb2fca => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:31:23.492424 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e74fb9-568a-4cf5-8c6d-6918ae562c78 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:23.495160 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-84e74fb9-568a-4cf5-8c6d-6918ae562c78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:23.549802 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:23.604268 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e74fb9-568a-4cf5-8c6d-6918ae562c78 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:23.620230 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc97839-ff8a-4bdd-8186-ca98ad33c92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:23.623178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc97839-ff8a-4bdd-8186-ca98ad33c92a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:23.624108 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc97839-ff8a-4bdd-8186-ca98ad33c92a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:23.625623 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc97839-ff8a-4bdd-8186-ca98ad33c92a admin admin] PATH_INFO: `/v3/users/7ac21fd2138e47a5abfb7408a6d84002` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:23.628591 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc97839-ff8a-4bdd-8186-ca98ad33c92a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ac21fd2138e47a5abfb7408a6d84002)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:23.631105 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc97839-ff8a-4bdd-8186-ca98ad33c92a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:23.631984 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d499d-129e-4a82-8f48-a24b613566ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:23.632814 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d499d-129e-4a82-8f48-a24b613566ae service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:23.633150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d499d-129e-4a82-8f48-a24b613566ae service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:23.635070 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d499d-129e-4a82-8f48-a24b613566ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:23.639785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-919d499d-129e-4a82-8f48-a24b613566ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:23.669669 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919d499d-129e-4a82-8f48-a24b613566ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:23.671906 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919d499d-129e-4a82-8f48-a24b613566ae service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:23.735466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 910/1798] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:31:23.775694 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9fc97839-ff8a-4bdd-8186-ca98ad33c92a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ac21fd2138e47a5abfb7408a6d84002'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:23.791012 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9fc97839-ff8a-4bdd-8186-ca98ad33c92a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ac21fd2138e47a5abfb7408a6d84002'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:23.792574 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9fc97839-ff8a-4bdd-8186-ca98ad33c92a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ac21fd2138e47a5abfb7408a6d84002'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:23.829697 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9fc97839-ff8a-4bdd-8186-ca98ad33c92a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ac21fd2138e47a5abfb7408a6d84002'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:23.852140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9fc97839-ff8a-4bdd-8186-ca98ad33c92a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ac21fd2138e47a5abfb7408a6d84002'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:23.886893 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 889/1799] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:31:23 2022] DELETE /identity/v3/users/7ac21fd2138e47a5abfb7408a6d84002 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:23.970446 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d499d-129e-4a82-8f48-a24b613566ae service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:23.973795 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-919d499d-129e-4a82-8f48-a24b613566ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:24.048601 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d499d-129e-4a82-8f48-a24b613566ae service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:24.189565 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:24.255855 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242af145-7564-4b77-9c56-7de607a7a58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:24.257202 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242af145-7564-4b77-9c56-7de607a7a58b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:24.257950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242af145-7564-4b77-9c56-7de607a7a58b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:24.258476 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242af145-7564-4b77-9c56-7de607a7a58b admin admin] PATH_INFO: `/v3/projects/80d47cc2ac594110b9187f6cfbdfbff6` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:24.260344 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242af145-7564-4b77-9c56-7de607a7a58b admin admin] RBAC: Authorizing `identity:delete_project(project_id=80d47cc2ac594110b9187f6cfbdfbff6)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:24.262619 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242af145-7564-4b77-9c56-7de607a7a58b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:24.381412 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-242af145-7564-4b77-9c56-7de607a7a58b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80d47cc2ac594110b9187f6cfbdfbff6', 'request_id': 'req-242af145-7564-4b77-9c56-7de607a7a58b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:24.396267 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-242af145-7564-4b77-9c56-7de607a7a58b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80d47cc2ac594110b9187f6cfbdfbff6', 'request_id': 'req-242af145-7564-4b77-9c56-7de607a7a58b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:24.417517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-242af145-7564-4b77-9c56-7de607a7a58b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80d47cc2ac594110b9187f6cfbdfbff6', 'request_id': 'req-242af145-7564-4b77-9c56-7de607a7a58b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:24.452609 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-242af145-7564-4b77-9c56-7de607a7a58b admin admin] The token cache is being invalidate because project 80d47cc2ac594110b9187f6cfbdfbff6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:31:24.452905 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-242af145-7564-4b77-9c56-7de607a7a58b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:24.454058 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-242af145-7564-4b77-9c56-7de607a7a58b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:24.459204 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 911/1800] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:31:23 2022] DELETE /identity/v3/projects/80d47cc2ac594110b9187f6cfbdfbff6 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:24.463017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc97839-ff8a-4bdd-8186-ca98ad33c92a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:24.465098 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9fc97839-ff8a-4bdd-8186-ca98ad33c92a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:24.480426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc97839-ff8a-4bdd-8186-ca98ad33c92a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:24.631763 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb879044-fd56-4bd6-9c54-6acaa52de879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:24.633120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb879044-fd56-4bd6-9c54-6acaa52de879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:24.633565 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb879044-fd56-4bd6-9c54-6acaa52de879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:24.634005 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb879044-fd56-4bd6-9c54-6acaa52de879 admin admin] PATH_INFO: `/v3/users/afa277d8e76448e693af064219fe484d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:24.636074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb879044-fd56-4bd6-9c54-6acaa52de879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afa277d8e76448e693af064219fe484d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:24.642631 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb879044-fd56-4bd6-9c54-6acaa52de879 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:24.738314 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:24.743002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bb879044-fd56-4bd6-9c54-6acaa52de879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afa277d8e76448e693af064219fe484d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:24.756457 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242af145-7564-4b77-9c56-7de607a7a58b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:24.758093 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-242af145-7564-4b77-9c56-7de607a7a58b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:24.763439 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bb879044-fd56-4bd6-9c54-6acaa52de879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afa277d8e76448e693af064219fe484d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:24.765258 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bb879044-fd56-4bd6-9c54-6acaa52de879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afa277d8e76448e693af064219fe484d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:24.777039 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bb879044-fd56-4bd6-9c54-6acaa52de879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afa277d8e76448e693af064219fe484d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:24.785231 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242af145-7564-4b77-9c56-7de607a7a58b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:24.802825 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bb879044-fd56-4bd6-9c54-6acaa52de879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afa277d8e76448e693af064219fe484d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:24.803740 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473e7d7-d4c5-44b8-8e9d-bee10e8d9b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:24.803740 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473e7d7-d4c5-44b8-8e9d-bee10e8d9b5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:24.803740 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473e7d7-d4c5-44b8-8e9d-bee10e8d9b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:24.803740 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473e7d7-d4c5-44b8-8e9d-bee10e8d9b5b admin admin] PATH_INFO: `/v3/projects/c734b732fc3d4844b70c3035e11350e2` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:24.805826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473e7d7-d4c5-44b8-8e9d-bee10e8d9b5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c734b732fc3d4844b70c3035e11350e2)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:24.806615 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473e7d7-d4c5-44b8-8e9d-bee10e8d9b5b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:24.815647 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 890/1801] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:31:24 2022] DELETE /identity/v3/users/afa277d8e76448e693af064219fe484d => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:24.872747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e473e7d7-d4c5-44b8-8e9d-bee10e8d9b5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c734b732fc3d4844b70c3035e11350e2', 'request_id': 'req-e473e7d7-d4c5-44b8-8e9d-bee10e8d9b5b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:24.881007 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e473e7d7-d4c5-44b8-8e9d-bee10e8d9b5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c734b732fc3d4844b70c3035e11350e2', 'request_id': 'req-e473e7d7-d4c5-44b8-8e9d-bee10e8d9b5b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:24.888405 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e473e7d7-d4c5-44b8-8e9d-bee10e8d9b5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c734b732fc3d4844b70c3035e11350e2', 'request_id': 'req-e473e7d7-d4c5-44b8-8e9d-bee10e8d9b5b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:24.913722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e473e7d7-d4c5-44b8-8e9d-bee10e8d9b5b admin admin] The token cache is being invalidate because project c734b732fc3d4844b70c3035e11350e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:31:24.914001 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e473e7d7-d4c5-44b8-8e9d-bee10e8d9b5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:24.914852 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e473e7d7-d4c5-44b8-8e9d-bee10e8d9b5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:24.920063 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 912/1802] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:31:24 2022] DELETE /identity/v3/projects/c734b732fc3d4844b70c3035e11350e2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:24.924771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e7ad2-6ce2-47e7-b92f-6832cb4ab396 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:24.925056 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e7ad2-6ce2-47e7-b92f-6832cb4ab396 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:24.925327 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e7ad2-6ce2-47e7-b92f-6832cb4ab396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:24.947369 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-276e7ad2-6ce2-47e7-b92f-6832cb4ab396 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:24.948887 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-276e7ad2-6ce2-47e7-b92f-6832cb4ab396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:25.044676 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284720a-8530-4619-ab47-bfd4d02415e6 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:25.045174 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284720a-8530-4619-ab47-bfd4d02415e6 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:25.045442 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284720a-8530-4619-ab47-bfd4d02415e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:25.049569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 891/1803] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:25.075358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-3284720a-8530-4619-ab47-bfd4d02415e6 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:25.077199 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3284720a-8530-4619-ab47-bfd4d02415e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:25.086058 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 913/1804] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:25.090078 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e7ad2-6ce2-47e7-b92f-6832cb4ab396 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:25.104869 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e7ad2-6ce2-47e7-b92f-6832cb4ab396 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:25.110658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb45dc-4ff4-4dec-8e4f-253eb3960246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:25.111501 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb45dc-4ff4-4dec-8e4f-253eb3960246 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:25.111655 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb45dc-4ff4-4dec-8e4f-253eb3960246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:25.111875 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb45dc-4ff4-4dec-8e4f-253eb3960246 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:25.112753 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb45dc-4ff4-4dec-8e4f-253eb3960246 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:31:25.112871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb45dc-4ff4-4dec-8e4f-253eb3960246 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:25.115505 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb45dc-4ff4-4dec-8e4f-253eb3960246 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:25.129862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 892/1805] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:31:25.134680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284720a-8530-4619-ab47-bfd4d02415e6 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:25.159809 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284720a-8530-4619-ab47-bfd4d02415e6 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:25.166159 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c21fc-9f6a-4274-8da8-5fda7e384ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:25.169428 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c21fc-9f6a-4274-8da8-5fda7e384ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:25.169688 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c21fc-9f6a-4274-8da8-5fda7e384ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:25.169967 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c21fc-9f6a-4274-8da8-5fda7e384ae7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:25.170667 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c21fc-9f6a-4274-8da8-5fda7e384ae7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:25.173624 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c21fc-9f6a-4274-8da8-5fda7e384ae7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:25.198281 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb45dc-4ff4-4dec-8e4f-253eb3960246 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:25.200759 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-61eb45dc-4ff4-4dec-8e4f-253eb3960246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:25.204041 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 914/1806] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:31:25 2022] POST /identity/v3/projects => generated 369 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:25.208487 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c21fc-9f6a-4274-8da8-5fda7e384ae7 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:25.215189 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb45dc-4ff4-4dec-8e4f-253eb3960246 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:25.243804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:25.294802 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ea107-4ef5-492b-abd4-2eb18d15b51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:25.295434 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ea107-4ef5-492b-abd4-2eb18d15b51d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:25.295557 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c21fc-9f6a-4274-8da8-5fda7e384ae7 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:25.295714 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ea107-4ef5-492b-abd4-2eb18d15b51d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:25.295987 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ea107-4ef5-492b-abd4-2eb18d15b51d admin admin] PATH_INFO: `/v3/projects/4ee5ddb8080648edaae0f0ff46da4d21` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:25.297105 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ea107-4ef5-492b-abd4-2eb18d15b51d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ee5ddb8080648edaae0f0ff46da4d21)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:25.299097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ea107-4ef5-492b-abd4-2eb18d15b51d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:25.301934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d4297-7d37-4813-8d53-7badddd6d34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:25.302674 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d4297-7d37-4813-8d53-7badddd6d34f admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:25.302958 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d4297-7d37-4813-8d53-7badddd6d34f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:25.303240 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d4297-7d37-4813-8d53-7badddd6d34f admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:25.304283 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d4297-7d37-4813-8d53-7badddd6d34f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:25.307759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d4297-7d37-4813-8d53-7badddd6d34f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:25.339973 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 915/1807] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:31:25 2022] POST /identity/v3/users => generated 285 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:25.351454 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d4297-7d37-4813-8d53-7badddd6d34f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:25.364890 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d12ea107-4ef5-492b-abd4-2eb18d15b51d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ee5ddb8080648edaae0f0ff46da4d21', 'request_id': 'req-d12ea107-4ef5-492b-abd4-2eb18d15b51d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:25.367685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:25.369257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d4297-7d37-4813-8d53-7badddd6d34f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:25.372919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d12ea107-4ef5-492b-abd4-2eb18d15b51d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ee5ddb8080648edaae0f0ff46da4d21', 'request_id': 'req-d12ea107-4ef5-492b-abd4-2eb18d15b51d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:25.384393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d12ea107-4ef5-492b-abd4-2eb18d15b51d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ee5ddb8080648edaae0f0ff46da4d21', 'request_id': 'req-d12ea107-4ef5-492b-abd4-2eb18d15b51d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:25.397315 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d12ea107-4ef5-492b-abd4-2eb18d15b51d admin admin] The token cache is being invalidate because project 4ee5ddb8080648edaae0f0ff46da4d21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:31:25.397629 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d12ea107-4ef5-492b-abd4-2eb18d15b51d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:25.398364 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d12ea107-4ef5-492b-abd4-2eb18d15b51d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:25.400322 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 893/1808] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:31:25 2022] DELETE /identity/v3/projects/4ee5ddb8080648edaae0f0ff46da4d21 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:31:25.405917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1c5ce-ed79-415b-bd2f-6e339d5c0232 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:25.406304 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1c5ce-ed79-415b-bd2f-6e339d5c0232 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:25.406669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1c5ce-ed79-415b-bd2f-6e339d5c0232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:25.426907 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-02e1c5ce-ed79-415b-bd2f-6e339d5c0232 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:25.428117 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-02e1c5ce-ed79-415b-bd2f-6e339d5c0232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:25.445595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed7dad6-5d7d-4cb0-a2db-411d761a8662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:25.446231 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7dad6-5d7d-4cb0-a2db-411d761a8662 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:25.446518 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7dad6-5d7d-4cb0-a2db-411d761a8662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:25.447110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7dad6-5d7d-4cb0-a2db-411d761a8662 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:25.447664 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed7dad6-5d7d-4cb0-a2db-411d761a8662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:25.453605 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed7dad6-5d7d-4cb0-a2db-411d761a8662 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:25.466947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 916/1809] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:25.472373 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed7dad6-5d7d-4cb0-a2db-411d761a8662 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:25.474712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2ed7dad6-5d7d-4cb0-a2db-411d761a8662 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:25.494024 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed7dad6-5d7d-4cb0-a2db-411d761a8662 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:25.495201 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 894/1810] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:31:25.500861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c7b25-c731-41a4-87ed-f38e64db00b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:25.501563 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c7b25-c731-41a4-87ed-f38e64db00b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:25.502019 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c7b25-c731-41a4-87ed-f38e64db00b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:25.502549 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c7b25-c731-41a4-87ed-f38e64db00b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:25.502704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80a539-6a5c-49ef-94bf-cfbe703548d9 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:25.502994 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80a539-6a5c-49ef-94bf-cfbe703548d9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:25.503270 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80a539-6a5c-49ef-94bf-cfbe703548d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:25.503707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7c7b25-c731-41a4-87ed-f38e64db00b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:25.505927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7c7b25-c731-41a4-87ed-f38e64db00b5 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:25.525221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 917/1811] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:31:25.525400 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ae80a539-6a5c-49ef-94bf-cfbe703548d9 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:25.525400 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ae80a539-6a5c-49ef-94bf-cfbe703548d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:25.526981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c7b25-c731-41a4-87ed-f38e64db00b5 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:25.532961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 895/1812] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:31:25.537273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80a539-6a5c-49ef-94bf-cfbe703548d9 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:25.537698 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c7b25-c731-41a4-87ed-f38e64db00b5 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:25.542567 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f877465-b2cd-411f-9c19-61672fbc98bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:25.543290 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f877465-b2cd-411f-9c19-61672fbc98bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:25.543613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f877465-b2cd-411f-9c19-61672fbc98bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:25.543846 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f877465-b2cd-411f-9c19-61672fbc98bc admin admin] PATH_INFO: `/v3/projects/583cd2b4f1ed44aaa8b360870ef5ea9a/users/02befbba4787418c9ade7413b43ea644/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:25.546667 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80a539-6a5c-49ef-94bf-cfbe703548d9 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:25.552361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d3a51-c1f3-40f8-8700-b6d1cb9d5f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:25.553220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d3a51-c1f3-40f8-8700-b6d1cb9d5f07 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:25.553220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d3a51-c1f3-40f8-8700-b6d1cb9d5f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:25.553562 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d3a51-c1f3-40f8-8700-b6d1cb9d5f07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:25.554151 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d3a51-c1f3-40f8-8700-b6d1cb9d5f07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:31:25.554509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d3a51-c1f3-40f8-8700-b6d1cb9d5f07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:25.556110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d3a51-c1f3-40f8-8700-b6d1cb9d5f07 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:25.561613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f877465-b2cd-411f-9c19-61672fbc98bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=583cd2b4f1ed44aaa8b360870ef5ea9a, user_id=02befbba4787418c9ade7413b43ea644, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:25.564831 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f877465-b2cd-411f-9c19-61672fbc98bc admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:25.573293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 896/1813] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:31:25.578324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d3a51-c1f3-40f8-8700-b6d1cb9d5f07 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:25.586060 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 918/1814] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:25 2022] PUT /identity/v3/projects/583cd2b4f1ed44aaa8b360870ef5ea9a/users/02befbba4787418c9ade7413b43ea644/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:25.593436 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a942b9-4443-4b0c-b961-b1bd3acece75 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:25.593940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a942b9-4443-4b0c-b961-b1bd3acece75 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:25.594390 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a942b9-4443-4b0c-b961-b1bd3acece75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:25.594805 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d3a51-c1f3-40f8-8700-b6d1cb9d5f07 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:25.656908 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-25a942b9-4443-4b0c-b961-b1bd3acece75 None None] MFA Rules not processed for user `02befbba4787418c9ade7413b43ea644`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:25.658986 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-25a942b9-4443-4b0c-b961-b1bd3acece75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:25.694509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3158d5f0-c8db-4b87-a8f2-8a7dcea438f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:25.695352 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158d5f0-c8db-4b87-a8f2-8a7dcea438f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:25.695821 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158d5f0-c8db-4b87-a8f2-8a7dcea438f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:25.696133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158d5f0-c8db-4b87-a8f2-8a7dcea438f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:25.697174 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3158d5f0-c8db-4b87-a8f2-8a7dcea438f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:25.698753 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3158d5f0-c8db-4b87-a8f2-8a7dcea438f2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:25.747394 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 897/1815] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:31:25 2022] POST /identity/v3/projects => generated 385 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:31:25.750556 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3158d5f0-c8db-4b87-a8f2-8a7dcea438f2 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:25.792763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:25.800512 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 919/1816] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:25.808274 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a942b9-4443-4b0c-b961-b1bd3acece75 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:25.830046 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a942b9-4443-4b0c-b961-b1bd3acece75 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:25.837003 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:25.866057 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3158d5f0-c8db-4b87-a8f2-8a7dcea438f2 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:25.873869 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b90c4-4ad8-41a2-b299-ad4ec2ebf776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:25.874742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b90c4-4ad8-41a2-b299-ad4ec2ebf776 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:25.875142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b90c4-4ad8-41a2-b299-ad4ec2ebf776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:25.875520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b90c4-4ad8-41a2-b299-ad4ec2ebf776 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:25.876456 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b90c4-4ad8-41a2-b299-ad4ec2ebf776 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:25.878783 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b90c4-4ad8-41a2-b299-ad4ec2ebf776 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:25.910419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 898/1817] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:31:25 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:31:25.912189 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b90c4-4ad8-41a2-b299-ad4ec2ebf776 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:25.914669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ae2b90c4-4ad8-41a2-b299-ad4ec2ebf776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:25.929490 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b90c4-4ad8-41a2-b299-ad4ec2ebf776 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:25.935519 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc63e7-55c0-4397-8a63-b5c1870891a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:25.936307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc63e7-55c0-4397-8a63-b5c1870891a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:25.936691 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc63e7-55c0-4397-8a63-b5c1870891a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:25.937086 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc63e7-55c0-4397-8a63-b5c1870891a5 admin admin] PATH_INFO: `/v3/users/e453798028f84a92af8b34238c129065` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:25.938589 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc63e7-55c0-4397-8a63-b5c1870891a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e453798028f84a92af8b34238c129065)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:25.940915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc63e7-55c0-4397-8a63-b5c1870891a5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:25.996756 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1e47f-8107-4aab-a004-fca423cf5868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:25.998220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1e47f-8107-4aab-a004-fca423cf5868 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:25.998220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1e47f-8107-4aab-a004-fca423cf5868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:25.998220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1e47f-8107-4aab-a004-fca423cf5868 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:26.004216 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1e47f-8107-4aab-a004-fca423cf5868 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:26.006298 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1e47f-8107-4aab-a004-fca423cf5868 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:26.016051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8fbc63e7-55c0-4397-8a63-b5c1870891a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e453798028f84a92af8b34238c129065'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:26.025888 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8fbc63e7-55c0-4397-8a63-b5c1870891a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e453798028f84a92af8b34238c129065'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:26.027025 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8fbc63e7-55c0-4397-8a63-b5c1870891a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e453798028f84a92af8b34238c129065'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:26.040020 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8fbc63e7-55c0-4397-8a63-b5c1870891a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e453798028f84a92af8b34238c129065'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:26.068738 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8fbc63e7-55c0-4397-8a63-b5c1870891a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e453798028f84a92af8b34238c129065'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:26.076711 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 920/1818] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:31:25 2022] POST /identity/v3/projects => generated 367 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:26.077917 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1e47f-8107-4aab-a004-fca423cf5868 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:26.082288 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 899/1819] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:31:25 2022] DELETE /identity/v3/users/e453798028f84a92af8b34238c129065 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:26.083221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-17a1e47f-8107-4aab-a004-fca423cf5868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:26.084048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc63e7-55c0-4397-8a63-b5c1870891a5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:26.088208 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8fbc63e7-55c0-4397-8a63-b5c1870891a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:26.187866 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc63e7-55c0-4397-8a63-b5c1870891a5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:26.194711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58280d35-12e7-4cdd-b593-ef782fa16a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:26.195356 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58280d35-12e7-4cdd-b593-ef782fa16a10 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:26.195638 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58280d35-12e7-4cdd-b593-ef782fa16a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:26.195915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58280d35-12e7-4cdd-b593-ef782fa16a10 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:26.196567 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58280d35-12e7-4cdd-b593-ef782fa16a10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:26.199392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58280d35-12e7-4cdd-b593-ef782fa16a10 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:26.237606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 900/1820] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:31:26 2022] POST /identity/v3/users => generated 290 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:26.237606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403bb95-d323-4f43-a0cb-945bd3e66f18 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:26.237606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403bb95-d323-4f43-a0cb-945bd3e66f18 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:26.237606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403bb95-d323-4f43-a0cb-945bd3e66f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:26.251449 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1e47f-8107-4aab-a004-fca423cf5868 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:26.269656 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccece017-b743-46dd-a263-90ac0b434215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:26.271337 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccece017-b743-46dd-a263-90ac0b434215 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:26.271780 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccece017-b743-46dd-a263-90ac0b434215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:26.272629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccece017-b743-46dd-a263-90ac0b434215 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:26.274221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccece017-b743-46dd-a263-90ac0b434215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:26.275241 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0403bb95-d323-4f43-a0cb-945bd3e66f18 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:26.276547 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0403bb95-d323-4f43-a0cb-945bd3e66f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:26.277459 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccece017-b743-46dd-a263-90ac0b434215 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:26.295314 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 901/1821] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:26.295314 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403bb95-d323-4f43-a0cb-945bd3e66f18 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:26.305786 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403bb95-d323-4f43-a0cb-945bd3e66f18 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:26.307728 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 921/1822] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:26.309579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccece017-b743-46dd-a263-90ac0b434215 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:26.312999 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ccece017-b743-46dd-a263-90ac0b434215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:26.322395 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f975109-61e6-4a98-9760-cb92020ed6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:26.324142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f975109-61e6-4a98-9760-cb92020ed6f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:26.327164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f975109-61e6-4a98-9760-cb92020ed6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:26.327706 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f975109-61e6-4a98-9760-cb92020ed6f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:26.329351 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f975109-61e6-4a98-9760-cb92020ed6f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:26.331450 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f975109-61e6-4a98-9760-cb92020ed6f2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:26.346826 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 902/1823] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:26.348028 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f975109-61e6-4a98-9760-cb92020ed6f2 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:26.360716 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f975109-61e6-4a98-9760-cb92020ed6f2 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:26.366235 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ef01e-2244-4eb3-af2d-1295b080d6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:26.366875 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ef01e-2244-4eb3-af2d-1295b080d6f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:26.367152 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ef01e-2244-4eb3-af2d-1295b080d6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:26.367426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ef01e-2244-4eb3-af2d-1295b080d6f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:26.368147 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ef01e-2244-4eb3-af2d-1295b080d6f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:26.369815 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ef01e-2244-4eb3-af2d-1295b080d6f1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:26.383724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 903/1824] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:26.385135 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ef01e-2244-4eb3-af2d-1295b080d6f1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:26.422076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ef01e-2244-4eb3-af2d-1295b080d6f1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:26.425812 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccece017-b743-46dd-a263-90ac0b434215 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:26.429598 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef4062-c657-4b20-b386-f511ec6506ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:26.430279 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef4062-c657-4b20-b386-f511ec6506ec service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:26.430879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef4062-c657-4b20-b386-f511ec6506ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:26.431158 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef4062-c657-4b20-b386-f511ec6506ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:26.432146 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a8cf2-f082-4d00-af54-ca391b346e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:26.432967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a8cf2-f082-4d00-af54-ca391b346e6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:26.433256 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a8cf2-f082-4d00-af54-ca391b346e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:26.433533 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a8cf2-f082-4d00-af54-ca391b346e6b admin admin] PATH_INFO: `/v3/projects/12178b7270c84a6aa14e43ef19eb7bad/users/748fe541d54d49f896f735f3d7f350fd/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:26.439232 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=83,keystone:INSERT=7,keystone:DELETE=10 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:26.462187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef4062-c657-4b20-b386-f511ec6506ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:26.463885 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a8cf2-f082-4d00-af54-ca391b346e6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=12178b7270c84a6aa14e43ef19eb7bad, user_id=748fe541d54d49f896f735f3d7f350fd, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:26.464425 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef4062-c657-4b20-b386-f511ec6506ec service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:26.472554 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a8cf2-f082-4d00-af54-ca391b346e6b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:26.482642 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 922/1825] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:26.483421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef4062-c657-4b20-b386-f511ec6506ec service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:26.494686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 904/1826] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:26 2022] PUT /identity/v3/projects/12178b7270c84a6aa14e43ef19eb7bad/users/748fe541d54d49f896f735f3d7f350fd/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:26.494817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef4062-c657-4b20-b386-f511ec6506ec service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:26.500422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a8cf2-f082-4d00-af54-ca391b346e6b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:26.533838 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:26.620574 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a8cf2-f082-4d00-af54-ca391b346e6b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:26.625820 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166720e-9651-4160-bcac-0e7e0cfcb4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:26.626858 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166720e-9651-4160-bcac-0e7e0cfcb4b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:26.627142 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166720e-9651-4160-bcac-0e7e0cfcb4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:26.627590 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166720e-9651-4160-bcac-0e7e0cfcb4b2 admin admin] PATH_INFO: `/v3/projects/f9d10651ad0f4bd5a57c6df3c28878f2/users/0468f774ea7947099579aad6f50bddfb/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:26.632465 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4170b-2aaa-40d9-8a50-a98ca80d203f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:26.634126 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4170b-2aaa-40d9-8a50-a98ca80d203f admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:26.634725 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4170b-2aaa-40d9-8a50-a98ca80d203f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:26.635360 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4170b-2aaa-40d9-8a50-a98ca80d203f admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:26.636243 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4170b-2aaa-40d9-8a50-a98ca80d203f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:26.639244 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4170b-2aaa-40d9-8a50-a98ca80d203f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:26.657720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166720e-9651-4160-bcac-0e7e0cfcb4b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9d10651ad0f4bd5a57c6df3c28878f2, user_id=0468f774ea7947099579aad6f50bddfb, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:26.659926 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 905/1827] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:26.660940 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4170b-2aaa-40d9-8a50-a98ca80d203f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:26.675914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166720e-9651-4160-bcac-0e7e0cfcb4b2 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:26.678746 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4170b-2aaa-40d9-8a50-a98ca80d203f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:26.687362 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0cfef-0d8b-463b-8e81-1db2827e9892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:26.688199 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0cfef-0d8b-463b-8e81-1db2827e9892 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:26.688505 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0cfef-0d8b-463b-8e81-1db2827e9892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:26.688998 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0cfef-0d8b-463b-8e81-1db2827e9892 admin admin] PATH_INFO: `/v3/projects/12178b7270c84a6aa14e43ef19eb7bad/users/748fe541d54d49f896f735f3d7f350fd/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:26.691606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e0cfef-0d8b-463b-8e81-1db2827e9892 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12178b7270c84a6aa14e43ef19eb7bad, user_id=748fe541d54d49f896f735f3d7f350fd, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:26.697238 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e0cfef-0d8b-463b-8e81-1db2827e9892 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:26.701846 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 923/1828] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:26 2022] PUT /identity/v3/projects/f9d10651ad0f4bd5a57c6df3c28878f2/users/0468f774ea7947099579aad6f50bddfb/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:26.714084 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56514c5-d2ca-4c2f-bdab-1615a12f20ff None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:26.714479 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56514c5-d2ca-4c2f-bdab-1615a12f20ff None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:26.715108 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56514c5-d2ca-4c2f-bdab-1615a12f20ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:26.719615 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 906/1829] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:26 2022] PUT /identity/v3/projects/12178b7270c84a6aa14e43ef19eb7bad/users/748fe541d54d49f896f735f3d7f350fd/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:26.736566 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd586f1-1053-450e-b6cd-4d62a2e940dc None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:26.736858 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd586f1-1053-450e-b6cd-4d62a2e940dc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:26.737137 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd586f1-1053-450e-b6cd-4d62a2e940dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:26.813147 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a56514c5-d2ca-4c2f-bdab-1615a12f20ff None None] MFA Rules not processed for user `0468f774ea7947099579aad6f50bddfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:26.814813 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a56514c5-d2ca-4c2f-bdab-1615a12f20ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:26.826653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1bd586f1-1053-450e-b6cd-4d62a2e940dc None None] MFA Rules not processed for user `748fe541d54d49f896f735f3d7f350fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:26.828392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1bd586f1-1053-450e-b6cd-4d62a2e940dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:26.930975 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 924/1830] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:26.937756 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56514c5-d2ca-4c2f-bdab-1615a12f20ff None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:26.971808 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 907/1831] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:26 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:26.983041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd586f1-1053-450e-b6cd-4d62a2e940dc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:26.985219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1bd586f1-1053-450e-b6cd-4d62a2e940dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:27.068854 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd586f1-1053-450e-b6cd-4d62a2e940dc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:27.072849 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56514c5-d2ca-4c2f-bdab-1615a12f20ff None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:27.073593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a36ef-1896-4ce8-982d-c69021c7888c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:27.074402 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a36ef-1896-4ce8-982d-c69021c7888c service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:27.074829 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a36ef-1896-4ce8-982d-c69021c7888c service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:27.075388 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a36ef-1896-4ce8-982d-c69021c7888c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:27.084880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deabf4e-78ce-42d2-9867-f1758cd4d169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:27.085844 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:27.086934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deabf4e-78ce-42d2-9867-f1758cd4d169 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:27.087382 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deabf4e-78ce-42d2-9867-f1758cd4d169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:27.087853 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deabf4e-78ce-42d2-9867-f1758cd4d169 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:27.088685 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deabf4e-78ce-42d2-9867-f1758cd4d169 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:27.089213 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a36ef-1896-4ce8-982d-c69021c7888c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:27.092128 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deabf4e-78ce-42d2-9867-f1758cd4d169 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:27.092516 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a36ef-1896-4ce8-982d-c69021c7888c service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:27.133131 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 925/1832] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:31:26 2022] POST /identity/v3/projects => generated 385 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:27.134568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 908/1833] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:26 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:31:27.134678 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deabf4e-78ce-42d2-9867-f1758cd4d169 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:27.169722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:27.225731 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deabf4e-78ce-42d2-9867-f1758cd4d169 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:27.232459 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a9d697-4bb8-40cc-8b55-ee8e00412ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:27.233947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9d697-4bb8-40cc-8b55-ee8e00412ce8 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:27.234338 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9d697-4bb8-40cc-8b55-ee8e00412ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:27.234338 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9d697-4bb8-40cc-8b55-ee8e00412ce8 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:27.234338 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a9d697-4bb8-40cc-8b55-ee8e00412ce8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:27.234338 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a9d697-4bb8-40cc-8b55-ee8e00412ce8 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:27.272880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 926/1834] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:31:27 2022] POST /identity/v3/users => generated 293 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:27.277386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a36ef-1896-4ce8-982d-c69021c7888c service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:27.292341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a36ef-1896-4ce8-982d-c69021c7888c service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:27.301539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e6d4e-bcfc-448e-8e95-98eb2a1f1c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:27.302081 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e6d4e-bcfc-448e-8e95-98eb2a1f1c35 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:27.302533 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e6d4e-bcfc-448e-8e95-98eb2a1f1c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:27.302873 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e6d4e-bcfc-448e-8e95-98eb2a1f1c35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:27.304452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a9d697-4bb8-40cc-8b55-ee8e00412ce8 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:27.306911 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-31a9d697-4bb8-40cc-8b55-ee8e00412ce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:27.316404 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8e6d4e-bcfc-448e-8e95-98eb2a1f1c35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:27.322035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8e6d4e-bcfc-448e-8e95-98eb2a1f1c35 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:27.341669 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a9d697-4bb8-40cc-8b55-ee8e00412ce8 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:27.346034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 909/1835] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:27.348968 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e6d4e-bcfc-448e-8e95-98eb2a1f1c35 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:27.351249 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624bfe28-c18c-4bed-9af3-1a2ebfc4981e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:27.352381 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bfe28-c18c-4bed-9af3-1a2ebfc4981e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:27.353948 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bfe28-c18c-4bed-9af3-1a2ebfc4981e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:27.353948 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bfe28-c18c-4bed-9af3-1a2ebfc4981e admin admin] PATH_INFO: `/v3/projects/57c23332fe6e467d9200920e4d0fe593` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:27.355235 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624bfe28-c18c-4bed-9af3-1a2ebfc4981e admin admin] RBAC: Authorizing `identity:delete_project(project_id=57c23332fe6e467d9200920e4d0fe593)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:27.356946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624bfe28-c18c-4bed-9af3-1a2ebfc4981e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:27.369433 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e6d4e-bcfc-448e-8e95-98eb2a1f1c35 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:27.378781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2639a2e6-7d84-424a-ad3c-116272768736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:27.380050 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639a2e6-7d84-424a-ad3c-116272768736 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:27.380532 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639a2e6-7d84-424a-ad3c-116272768736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:27.381491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639a2e6-7d84-424a-ad3c-116272768736 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:27.382931 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2639a2e6-7d84-424a-ad3c-116272768736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:27.385463 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2639a2e6-7d84-424a-ad3c-116272768736 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:27.402495 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 910/1836] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:27.407354 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2639a2e6-7d84-424a-ad3c-116272768736 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:27.422219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2639a2e6-7d84-424a-ad3c-116272768736 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:27.429957 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-624bfe28-c18c-4bed-9af3-1a2ebfc4981e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57c23332fe6e467d9200920e4d0fe593', 'request_id': 'req-624bfe28-c18c-4bed-9af3-1a2ebfc4981e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:27.440482 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-624bfe28-c18c-4bed-9af3-1a2ebfc4981e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57c23332fe6e467d9200920e4d0fe593', 'request_id': 'req-624bfe28-c18c-4bed-9af3-1a2ebfc4981e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:27.446675 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-624bfe28-c18c-4bed-9af3-1a2ebfc4981e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57c23332fe6e467d9200920e4d0fe593', 'request_id': 'req-624bfe28-c18c-4bed-9af3-1a2ebfc4981e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:27.496310 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-624bfe28-c18c-4bed-9af3-1a2ebfc4981e admin admin] The token cache is being invalidate because project 57c23332fe6e467d9200920e4d0fe593 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:31:27.497322 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-624bfe28-c18c-4bed-9af3-1a2ebfc4981e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:27.498748 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-624bfe28-c18c-4bed-9af3-1a2ebfc4981e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:27.501711 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 927/1837] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:31:27 2022] DELETE /identity/v3/projects/57c23332fe6e467d9200920e4d0fe593 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:27.507977 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624bfe28-c18c-4bed-9af3-1a2ebfc4981e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:27.510674 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-624bfe28-c18c-4bed-9af3-1a2ebfc4981e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:27.585129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d111cb-33b0-408f-9596-f9df704ba536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:27.585812 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d111cb-33b0-408f-9596-f9df704ba536 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:27.586093 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d111cb-33b0-408f-9596-f9df704ba536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:27.586400 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d111cb-33b0-408f-9596-f9df704ba536 admin admin] PATH_INFO: `/v3/projects/63deaa5cbb0c41d29f986991c9e04020/users/854b04e7b10d4d739f50454c1cd38156/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:27.616431 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624bfe28-c18c-4bed-9af3-1a2ebfc4981e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:27.621503 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d111cb-33b0-408f-9596-f9df704ba536 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63deaa5cbb0c41d29f986991c9e04020, user_id=854b04e7b10d4d739f50454c1cd38156, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:27.624207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895ade3-5b50-4651-b7e9-f0a9fd1f392b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:27.625644 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d111cb-33b0-408f-9596-f9df704ba536 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:27.625890 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895ade3-5b50-4651-b7e9-f0a9fd1f392b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:27.626625 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895ade3-5b50-4651-b7e9-f0a9fd1f392b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:27.626625 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895ade3-5b50-4651-b7e9-f0a9fd1f392b admin admin] PATH_INFO: `/v3/users/b48ae6ee306d4f2b9539c5c0b2f29cc2` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:27.627816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895ade3-5b50-4651-b7e9-f0a9fd1f392b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b48ae6ee306d4f2b9539c5c0b2f29cc2)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:27.630899 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895ade3-5b50-4651-b7e9-f0a9fd1f392b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:27.656347 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 911/1838] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:27 2022] PUT /identity/v3/projects/63deaa5cbb0c41d29f986991c9e04020/users/854b04e7b10d4d739f50454c1cd38156/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:27.667462 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ff076-a5a5-46a4-a10d-f5233b10bb4d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:27.667863 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ff076-a5a5-46a4-a10d-f5233b10bb4d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:27.668217 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ff076-a5a5-46a4-a10d-f5233b10bb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:27.716258 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-996ff076-a5a5-46a4-a10d-f5233b10bb4d None None] MFA Rules not processed for user `854b04e7b10d4d739f50454c1cd38156`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:27.717527 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-996ff076-a5a5-46a4-a10d-f5233b10bb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:27.726043 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=84,keystone:INSERT=4,keystone:DELETE=12 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:27.747517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d895ade3-5b50-4651-b7e9-f0a9fd1f392b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b48ae6ee306d4f2b9539c5c0b2f29cc2'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:27.761036 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d895ade3-5b50-4651-b7e9-f0a9fd1f392b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b48ae6ee306d4f2b9539c5c0b2f29cc2'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:27.762600 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d895ade3-5b50-4651-b7e9-f0a9fd1f392b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b48ae6ee306d4f2b9539c5c0b2f29cc2'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:27.763339 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d895ade3-5b50-4651-b7e9-f0a9fd1f392b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b48ae6ee306d4f2b9539c5c0b2f29cc2'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:27.780846 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d895ade3-5b50-4651-b7e9-f0a9fd1f392b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b48ae6ee306d4f2b9539c5c0b2f29cc2'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:27.805416 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 928/1839] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:31:27 2022] DELETE /identity/v3/users/b48ae6ee306d4f2b9539c5c0b2f29cc2 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:27.827198 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 912/1840] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:31:27.832039 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895ade3-5b50-4651-b7e9-f0a9fd1f392b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:27.834792 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d895ade3-5b50-4651-b7e9-f0a9fd1f392b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:27.936963 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895ade3-5b50-4651-b7e9-f0a9fd1f392b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:27.944026 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348da0d-f718-4f60-91c8-ad3d4ca39058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:27.944992 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348da0d-f718-4f60-91c8-ad3d4ca39058 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:27.945455 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348da0d-f718-4f60-91c8-ad3d4ca39058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:27.945698 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348da0d-f718-4f60-91c8-ad3d4ca39058 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:27.946489 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2348da0d-f718-4f60-91c8-ad3d4ca39058 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:27.948577 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2348da0d-f718-4f60-91c8-ad3d4ca39058 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:27.999482 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 929/1841] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:31:27 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:31:28.003337 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ff076-a5a5-46a4-a10d-f5233b10bb4d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:28.065270 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:28.083885 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ff076-a5a5-46a4-a10d-f5233b10bb4d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:28.160220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c593e2-55e9-4947-9945-26e86b24a427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:28.162419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c593e2-55e9-4947-9945-26e86b24a427 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:28.162799 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c593e2-55e9-4947-9945-26e86b24a427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:28.163154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c593e2-55e9-4947-9945-26e86b24a427 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:28.163882 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c593e2-55e9-4947-9945-26e86b24a427 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:28.165883 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c593e2-55e9-4947-9945-26e86b24a427 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:28.193191 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 913/1842] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:31:28 2022] POST /identity/v3/users => generated 299 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:28.198388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348da0d-f718-4f60-91c8-ad3d4ca39058 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:28.210351 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348da0d-f718-4f60-91c8-ad3d4ca39058 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:28.215913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c593e2-55e9-4947-9945-26e86b24a427 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:28.217668 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-37c593e2-55e9-4947-9945-26e86b24a427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:28.225325 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c46b734-0604-4e50-98b7-e8b4ab4f4585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:28.225946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46b734-0604-4e50-98b7-e8b4ab4f4585 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:28.226292 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46b734-0604-4e50-98b7-e8b4ab4f4585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:28.226522 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46b734-0604-4e50-98b7-e8b4ab4f4585 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:28.227205 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c46b734-0604-4e50-98b7-e8b4ab4f4585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:28.228808 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c46b734-0604-4e50-98b7-e8b4ab4f4585 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:28.236905 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c593e2-55e9-4947-9945-26e86b24a427 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:28.244126 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea071c1-552c-4fe3-b56d-66f08def0d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:28.244988 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea071c1-552c-4fe3-b56d-66f08def0d19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:28.245342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea071c1-552c-4fe3-b56d-66f08def0d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:28.245627 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea071c1-552c-4fe3-b56d-66f08def0d19 admin admin] PATH_INFO: `/v3/projects/96dcecfa7b9b4c28b736860a392029fb` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:28.247540 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea071c1-552c-4fe3-b56d-66f08def0d19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96dcecfa7b9b4c28b736860a392029fb)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:28.249434 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea071c1-552c-4fe3-b56d-66f08def0d19 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:28.260754 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 930/1843] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:28.261647 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c46b734-0604-4e50-98b7-e8b4ab4f4585 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:28.272367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c46b734-0604-4e50-98b7-e8b4ab4f4585 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:28.280097 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a0042-86a2-470c-b775-c44cde2362a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:28.281985 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a0042-86a2-470c-b775-c44cde2362a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:28.282567 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a0042-86a2-470c-b775-c44cde2362a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:28.282913 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a0042-86a2-470c-b775-c44cde2362a2 admin admin] PATH_INFO: `/v3/projects/3e2c0f4d09d841f2b8550e611461ad6d/users/97b1e198cbc1435e8fab9fb9f9714391/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:28.303712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1a0042-86a2-470c-b775-c44cde2362a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e2c0f4d09d841f2b8550e611461ad6d, user_id=97b1e198cbc1435e8fab9fb9f9714391, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:28.306702 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1a0042-86a2-470c-b775-c44cde2362a2 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:28.321716 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9ea071c1-552c-4fe3-b56d-66f08def0d19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96dcecfa7b9b4c28b736860a392029fb', 'request_id': 'req-9ea071c1-552c-4fe3-b56d-66f08def0d19'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:28.329036 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 931/1844] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:28 2022] PUT /identity/v3/projects/3e2c0f4d09d841f2b8550e611461ad6d/users/97b1e198cbc1435e8fab9fb9f9714391/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:28.329182 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9ea071c1-552c-4fe3-b56d-66f08def0d19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96dcecfa7b9b4c28b736860a392029fb', 'request_id': 'req-9ea071c1-552c-4fe3-b56d-66f08def0d19'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:28.333539 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a0042-86a2-470c-b775-c44cde2362a2 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:28.339208 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9ea071c1-552c-4fe3-b56d-66f08def0d19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96dcecfa7b9b4c28b736860a392029fb', 'request_id': 'req-9ea071c1-552c-4fe3-b56d-66f08def0d19'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:28.353233 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9ea071c1-552c-4fe3-b56d-66f08def0d19 admin admin] The token cache is being invalidate because project 96dcecfa7b9b4c28b736860a392029fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:31:28.353524 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9ea071c1-552c-4fe3-b56d-66f08def0d19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:28.354181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9ea071c1-552c-4fe3-b56d-66f08def0d19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:28.356331 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 914/1845] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:31:28 2022] DELETE /identity/v3/projects/96dcecfa7b9b4c28b736860a392029fb => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:31:28.360303 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea071c1-552c-4fe3-b56d-66f08def0d19 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:28.361715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9ea071c1-552c-4fe3-b56d-66f08def0d19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:28.415304 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a0042-86a2-470c-b775-c44cde2362a2 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:28.420987 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ebd7b2-3275-4651-ab9d-9f19e650e930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:28.422293 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebd7b2-3275-4651-ab9d-9f19e650e930 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:28.422578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebd7b2-3275-4651-ab9d-9f19e650e930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:28.422981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebd7b2-3275-4651-ab9d-9f19e650e930 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:28.423705 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ebd7b2-3275-4651-ab9d-9f19e650e930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:28.425400 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ebd7b2-3275-4651-ab9d-9f19e650e930 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:28.430988 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:28.440115 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 932/1846] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:28.441570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ebd7b2-3275-4651-ab9d-9f19e650e930 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:28.443757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea071c1-552c-4fe3-b56d-66f08def0d19 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:28.444088 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-59ebd7b2-3275-4651-ab9d-9f19e650e930 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:28.455041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd077d-ea9e-4ec7-8728-aa62f660ccb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:28.456520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd077d-ea9e-4ec7-8728-aa62f660ccb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:28.456807 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd077d-ea9e-4ec7-8728-aa62f660ccb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:28.457081 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd077d-ea9e-4ec7-8728-aa62f660ccb4 admin admin] PATH_INFO: `/v3/users/32701be1bd624de6afe819d743d56c99` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:28.458356 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd077d-ea9e-4ec7-8728-aa62f660ccb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32701be1bd624de6afe819d743d56c99)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:28.460388 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd077d-ea9e-4ec7-8728-aa62f660ccb4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:28.468090 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ebd7b2-3275-4651-ab9d-9f19e650e930 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:28.468990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:28.477680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4c315-cf74-4b09-b21a-facd75c2dd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:28.478625 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4c315-cf74-4b09-b21a-facd75c2dd76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:28.478909 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4c315-cf74-4b09-b21a-facd75c2dd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:28.479185 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4c315-cf74-4b09-b21a-facd75c2dd76 admin admin] PATH_INFO: `/v3/projects/3e2c0f4d09d841f2b8550e611461ad6d/users/97b1e198cbc1435e8fab9fb9f9714391/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:28.484370 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e4c315-cf74-4b09-b21a-facd75c2dd76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e2c0f4d09d841f2b8550e611461ad6d, user_id=97b1e198cbc1435e8fab9fb9f9714391, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:28.487819 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e4c315-cf74-4b09-b21a-facd75c2dd76 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:28.506240 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 933/1847] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:28 2022] PUT /identity/v3/projects/3e2c0f4d09d841f2b8550e611461ad6d/users/97b1e198cbc1435e8fab9fb9f9714391/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:31:28.513091 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf433184-95aa-4d70-94bd-bd4d4fe538fa None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:28.514594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf433184-95aa-4d70-94bd-bd4d4fe538fa None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:28.515230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf433184-95aa-4d70-94bd-bd4d4fe538fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:28.540642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d7cd077d-ea9e-4ec7-8728-aa62f660ccb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32701be1bd624de6afe819d743d56c99'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:28.550049 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-bf433184-95aa-4d70-94bd-bd4d4fe538fa None None] MFA Rules not processed for user `97b1e198cbc1435e8fab9fb9f9714391`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:28.551290 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bf433184-95aa-4d70-94bd-bd4d4fe538fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:28.555035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d7cd077d-ea9e-4ec7-8728-aa62f660ccb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32701be1bd624de6afe819d743d56c99'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:28.555934 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d7cd077d-ea9e-4ec7-8728-aa62f660ccb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32701be1bd624de6afe819d743d56c99'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:28.568983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d7cd077d-ea9e-4ec7-8728-aa62f660ccb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32701be1bd624de6afe819d743d56c99'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:28.588550 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d7cd077d-ea9e-4ec7-8728-aa62f660ccb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32701be1bd624de6afe819d743d56c99'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:28.600018 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 915/1848] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:31:28 2022] DELETE /identity/v3/users/32701be1bd624de6afe819d743d56c99 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:31:28.643232 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 934/1849] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:28 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:31:28.653204 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd077d-ea9e-4ec7-8728-aa62f660ccb4 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:28.654698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d7cd077d-ea9e-4ec7-8728-aa62f660ccb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:28.820931 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd077d-ea9e-4ec7-8728-aa62f660ccb4 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:28.829443 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079942e9-ea7b-40eb-9dde-3f09f24892f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:28.831118 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079942e9-ea7b-40eb-9dde-3f09f24892f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:28.831479 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079942e9-ea7b-40eb-9dde-3f09f24892f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:28.831887 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079942e9-ea7b-40eb-9dde-3f09f24892f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:28.931152 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079942e9-ea7b-40eb-9dde-3f09f24892f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:28.933356 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079942e9-ea7b-40eb-9dde-3f09f24892f2 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:28.944968 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 916/1850] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:28 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:29.168681 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf433184-95aa-4d70-94bd-bd4d4fe538fa None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:29.170753 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bf433184-95aa-4d70-94bd-bd4d4fe538fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:29.244785 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079942e9-ea7b-40eb-9dde-3f09f24892f2 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:29.246383 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-079942e9-ea7b-40eb-9dde-3f09f24892f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:29.281330 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf433184-95aa-4d70-94bd-bd4d4fe538fa None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:29.306477 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:29.350480 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079942e9-ea7b-40eb-9dde-3f09f24892f2 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:29.355948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1586da-3040-4360-9cee-4a60d40e7367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:29.356784 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1586da-3040-4360-9cee-4a60d40e7367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:29.357133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1586da-3040-4360-9cee-4a60d40e7367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:29.357474 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1586da-3040-4360-9cee-4a60d40e7367 admin admin] PATH_INFO: `/v3/users/02befbba4787418c9ade7413b43ea644` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:29.365684 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1586da-3040-4360-9cee-4a60d40e7367 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02befbba4787418c9ade7413b43ea644)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:29.367938 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1586da-3040-4360-9cee-4a60d40e7367 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:29.376396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f8f9a-8aad-4f4a-ab53-635b0daa33ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:29.377078 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f8f9a-8aad-4f4a-ab53-635b0daa33ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:29.377421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f8f9a-8aad-4f4a-ab53-635b0daa33ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:29.377702 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f8f9a-8aad-4f4a-ab53-635b0daa33ac admin admin] PATH_INFO: `/v3/projects/eb5920d73c3a447181bc8ab452dc8924` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:29.379230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f8f9a-8aad-4f4a-ab53-635b0daa33ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb5920d73c3a447181bc8ab452dc8924)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:29.381179 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f8f9a-8aad-4f4a-ab53-635b0daa33ac admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:29.414452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:29.432831 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4b1586da-3040-4360-9cee-4a60d40e7367 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02befbba4787418c9ade7413b43ea644'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:29.440778 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4b1586da-3040-4360-9cee-4a60d40e7367 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02befbba4787418c9ade7413b43ea644'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:29.441467 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4b1586da-3040-4360-9cee-4a60d40e7367 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02befbba4787418c9ade7413b43ea644'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:29.443970 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-652f8f9a-8aad-4f4a-ab53-635b0daa33ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb5920d73c3a447181bc8ab452dc8924', 'request_id': 'req-652f8f9a-8aad-4f4a-ab53-635b0daa33ac'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:29.450328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4b1586da-3040-4360-9cee-4a60d40e7367 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02befbba4787418c9ade7413b43ea644'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:29.452233 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-652f8f9a-8aad-4f4a-ab53-635b0daa33ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb5920d73c3a447181bc8ab452dc8924', 'request_id': 'req-652f8f9a-8aad-4f4a-ab53-635b0daa33ac'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:29.462800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-652f8f9a-8aad-4f4a-ab53-635b0daa33ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb5920d73c3a447181bc8ab452dc8924', 'request_id': 'req-652f8f9a-8aad-4f4a-ab53-635b0daa33ac'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:29.465201 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4b1586da-3040-4360-9cee-4a60d40e7367 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02befbba4787418c9ade7413b43ea644'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:29.474473 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 917/1851] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:31:29 2022] DELETE /identity/v3/users/02befbba4787418c9ade7413b43ea644 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:29.480790 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efcb4b5-3925-46d9-a386-4f3a2599b214 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:29.481175 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efcb4b5-3925-46d9-a386-4f3a2599b214 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:29.481538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efcb4b5-3925-46d9-a386-4f3a2599b214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:29.484166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-652f8f9a-8aad-4f4a-ab53-635b0daa33ac admin admin] The token cache is being invalidate because project eb5920d73c3a447181bc8ab452dc8924 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:31:29.484558 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-652f8f9a-8aad-4f4a-ab53-635b0daa33ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:29.494361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-652f8f9a-8aad-4f4a-ab53-635b0daa33ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:29.494361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 935/1852] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:31:29 2022] DELETE /identity/v3/projects/eb5920d73c3a447181bc8ab452dc8924 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:29.494361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0bd7c-1f4b-4b7f-a735-280b4e17fe93 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:29.494361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0bd7c-1f4b-4b7f-a735-280b4e17fe93 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:29.494361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0bd7c-1f4b-4b7f-a735-280b4e17fe93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:29.508240 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8efcb4b5-3925-46d9-a386-4f3a2599b214 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:29.509456 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8efcb4b5-3925-46d9-a386-4f3a2599b214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:29.521894 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-bce0bd7c-1f4b-4b7f-a735-280b4e17fe93 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:29.523711 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bce0bd7c-1f4b-4b7f-a735-280b4e17fe93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:29.582331 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 918/1853] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:29.595563 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efcb4b5-3925-46d9-a386-4f3a2599b214 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:29.597945 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8efcb4b5-3925-46d9-a386-4f3a2599b214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:29.618137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 936/1854] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:31:29.624085 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220e503-b187-4d97-9bcc-b9b49d2b55e0 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:29.624499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220e503-b187-4d97-9bcc-b9b49d2b55e0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:29.624776 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220e503-b187-4d97-9bcc-b9b49d2b55e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:29.647709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7220e503-b187-4d97-9bcc-b9b49d2b55e0 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:29.649667 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7220e503-b187-4d97-9bcc-b9b49d2b55e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:29.658545 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 937/1855] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:29.662954 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220e503-b187-4d97-9bcc-b9b49d2b55e0 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:29.672922 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220e503-b187-4d97-9bcc-b9b49d2b55e0 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:29.677763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bfbd38-d49d-40c5-b371-92d9eb276819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:29.678506 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bfbd38-d49d-40c5-b371-92d9eb276819 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:29.678876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bfbd38-d49d-40c5-b371-92d9eb276819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:29.679116 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bfbd38-d49d-40c5-b371-92d9eb276819 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:29.680056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bfbd38-d49d-40c5-b371-92d9eb276819 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:31:29.680391 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bfbd38-d49d-40c5-b371-92d9eb276819 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:29.682065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bfbd38-d49d-40c5-b371-92d9eb276819 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:29.687062 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efcb4b5-3925-46d9-a386-4f3a2599b214 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:29.690906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e5ad4-9cfd-47b7-a37e-5ab06f1203c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:29.691519 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e5ad4-9cfd-47b7-a37e-5ab06f1203c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:29.691805 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e5ad4-9cfd-47b7-a37e-5ab06f1203c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:29.692078 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e5ad4-9cfd-47b7-a37e-5ab06f1203c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:29.698652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 938/1856] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:31:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:31:29.699343 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bfbd38-d49d-40c5-b371-92d9eb276819 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:29.709181 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-68bfbd38-d49d-40c5-b371-92d9eb276819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:29.716230 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e5ad4-9cfd-47b7-a37e-5ab06f1203c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:29.738933 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e5ad4-9cfd-47b7-a37e-5ab06f1203c3 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:29.747029 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:29.757168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 919/1857] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:31:29.758831 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e5ad4-9cfd-47b7-a37e-5ab06f1203c3 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:29.771660 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e5ad4-9cfd-47b7-a37e-5ab06f1203c3 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:29.777731 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc9f3d-2496-4d36-9109-b52754b21b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:29.778571 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc9f3d-2496-4d36-9109-b52754b21b1f admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:29.779006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc9f3d-2496-4d36-9109-b52754b21b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:29.779432 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc9f3d-2496-4d36-9109-b52754b21b1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:29.780430 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc9f3d-2496-4d36-9109-b52754b21b1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:29.783435 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc9f3d-2496-4d36-9109-b52754b21b1f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:29.807745 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bfbd38-d49d-40c5-b371-92d9eb276819 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:29.816271 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc199df-7b9b-4012-8f07-46351c0c56b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:29.817278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc199df-7b9b-4012-8f07-46351c0c56b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:29.817785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc199df-7b9b-4012-8f07-46351c0c56b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:29.818184 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc199df-7b9b-4012-8f07-46351c0c56b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:29.820289 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-cfc199df-7b9b-4012-8f07-46351c0c56b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:29.823562 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 920/1858] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:31:29 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:29.828768 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc9f3d-2496-4d36-9109-b52754b21b1f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:29.891911 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:29.923502 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc199df-7b9b-4012-8f07-46351c0c56b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:29.926623 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc199df-7b9b-4012-8f07-46351c0c56b8 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:29.939141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc9f3d-2496-4d36-9109-b52754b21b1f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:29.944212 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 939/1859] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:29.947128 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5797958-7376-49b7-a2cd-b2a765edef40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:29.948040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5797958-7376-49b7-a2cd-b2a765edef40 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:29.948551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5797958-7376-49b7-a2cd-b2a765edef40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:29.949080 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5797958-7376-49b7-a2cd-b2a765edef40 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:29.949964 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5797958-7376-49b7-a2cd-b2a765edef40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:29.959869 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5797958-7376-49b7-a2cd-b2a765edef40 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:30.016283 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 921/1860] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:31:29 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:30.020366 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc199df-7b9b-4012-8f07-46351c0c56b8 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:30.039704 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc199df-7b9b-4012-8f07-46351c0c56b8 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:30.047141 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd292af-70d0-4937-90fa-97fbf2fda105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:30.048145 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd292af-70d0-4937-90fa-97fbf2fda105 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:30.048631 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd292af-70d0-4937-90fa-97fbf2fda105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:30.049154 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd292af-70d0-4937-90fa-97fbf2fda105 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:30.050076 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd292af-70d0-4937-90fa-97fbf2fda105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:30.052190 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd292af-70d0-4937-90fa-97fbf2fda105 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:30.068622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 940/1861] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:30.071576 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5797958-7376-49b7-a2cd-b2a765edef40 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:30.087306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5797958-7376-49b7-a2cd-b2a765edef40 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:30.096726 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a2ae1-0fae-4ffd-8322-98419ff792e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:30.097439 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a2ae1-0fae-4ffd-8322-98419ff792e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:30.098268 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a2ae1-0fae-4ffd-8322-98419ff792e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:30.098268 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a2ae1-0fae-4ffd-8322-98419ff792e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:30.098761 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96a2ae1-0fae-4ffd-8322-98419ff792e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:30.100936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96a2ae1-0fae-4ffd-8322-98419ff792e8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:30.122762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 922/1862] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:30.124409 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd292af-70d0-4937-90fa-97fbf2fda105 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:30.136422 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd292af-70d0-4937-90fa-97fbf2fda105 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:30.144689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a239a1-f2c4-4fed-87e2-ebc5be71d783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:30.145781 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a239a1-f2c4-4fed-87e2-ebc5be71d783 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:30.146577 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a239a1-f2c4-4fed-87e2-ebc5be71d783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:30.146843 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a239a1-f2c4-4fed-87e2-ebc5be71d783 admin admin] PATH_INFO: `/v3/projects/a47490d8d86b4b0e92840d7479deb9ec/users/4d0657d18ffb4372ad084150a9cfd121/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:30.170574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a239a1-f2c4-4fed-87e2-ebc5be71d783 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a47490d8d86b4b0e92840d7479deb9ec, user_id=4d0657d18ffb4372ad084150a9cfd121, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:30.174687 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a239a1-f2c4-4fed-87e2-ebc5be71d783 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:30.196119 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 941/1863] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:30 2022] PUT /identity/v3/projects/a47490d8d86b4b0e92840d7479deb9ec/users/4d0657d18ffb4372ad084150a9cfd121/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:30.211050 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658d1f0-ce1f-4dda-9145-df0ef68e9e85 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:30.211410 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658d1f0-ce1f-4dda-9145-df0ef68e9e85 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:30.211705 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658d1f0-ce1f-4dda-9145-df0ef68e9e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:30.266517 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0658d1f0-ce1f-4dda-9145-df0ef68e9e85 None None] MFA Rules not processed for user `4d0657d18ffb4372ad084150a9cfd121`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:30.267781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0658d1f0-ce1f-4dda-9145-df0ef68e9e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:30.430457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 923/1864] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:30.443139 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a239a1-f2c4-4fed-87e2-ebc5be71d783 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:30.559204 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a239a1-f2c4-4fed-87e2-ebc5be71d783 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:30.570516 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5655f-3e2f-448b-a6aa-e865ea8ca21f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:30.571080 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5655f-3e2f-448b-a6aa-e865ea8ca21f service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:30.571365 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5655f-3e2f-448b-a6aa-e865ea8ca21f service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:30.571635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5655f-3e2f-448b-a6aa-e865ea8ca21f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:30.583945 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5655f-3e2f-448b-a6aa-e865ea8ca21f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:30.585504 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5655f-3e2f-448b-a6aa-e865ea8ca21f service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:30.597858 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 942/1865] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:30.603784 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0658d1f0-ce1f-4dda-9145-df0ef68e9e85 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:30.607395 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0658d1f0-ce1f-4dda-9145-df0ef68e9e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:30.631977 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5655f-3e2f-448b-a6aa-e865ea8ca21f service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:30.669182 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:30.690537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0658d1f0-ce1f-4dda-9145-df0ef68e9e85 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:30.696148 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e0f9a-f678-48e7-a61c-bb43dc9cd2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:30.697119 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e0f9a-f678-48e7-a61c-bb43dc9cd2c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:30.697471 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e0f9a-f678-48e7-a61c-bb43dc9cd2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:30.697719 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e0f9a-f678-48e7-a61c-bb43dc9cd2c5 admin admin] PATH_INFO: `/v3/projects/583cd2b4f1ed44aaa8b360870ef5ea9a` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:30.699099 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160e0f9a-f678-48e7-a61c-bb43dc9cd2c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=583cd2b4f1ed44aaa8b360870ef5ea9a)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:30.702350 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160e0f9a-f678-48e7-a61c-bb43dc9cd2c5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:30.713440 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5655f-3e2f-448b-a6aa-e865ea8ca21f service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:30.729316 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a9456-032a-43dc-af59-24424426288d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:30.730045 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a9456-032a-43dc-af59-24424426288d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:30.730663 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a9456-032a-43dc-af59-24424426288d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:30.731068 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a9456-032a-43dc-af59-24424426288d admin admin] PATH_INFO: `/v3/users/4d0657d18ffb4372ad084150a9cfd121` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:30.732597 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a9456-032a-43dc-af59-24424426288d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d0657d18ffb4372ad084150a9cfd121)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:30.734699 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a9456-032a-43dc-af59-24424426288d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:30.753078 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-160e0f9a-f678-48e7-a61c-bb43dc9cd2c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '583cd2b4f1ed44aaa8b360870ef5ea9a', 'request_id': 'req-160e0f9a-f678-48e7-a61c-bb43dc9cd2c5'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:30.759164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-160e0f9a-f678-48e7-a61c-bb43dc9cd2c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '583cd2b4f1ed44aaa8b360870ef5ea9a', 'request_id': 'req-160e0f9a-f678-48e7-a61c-bb43dc9cd2c5'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:30.767095 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-160e0f9a-f678-48e7-a61c-bb43dc9cd2c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '583cd2b4f1ed44aaa8b360870ef5ea9a', 'request_id': 'req-160e0f9a-f678-48e7-a61c-bb43dc9cd2c5'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:30.778134 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-160e0f9a-f678-48e7-a61c-bb43dc9cd2c5 admin admin] The token cache is being invalidate because project 583cd2b4f1ed44aaa8b360870ef5ea9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:31:30.778455 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-160e0f9a-f678-48e7-a61c-bb43dc9cd2c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:30.779109 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-160e0f9a-f678-48e7-a61c-bb43dc9cd2c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:30.781279 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 924/1866] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:31:30 2022] DELETE /identity/v3/projects/583cd2b4f1ed44aaa8b360870ef5ea9a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:30.785274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e0f9a-f678-48e7-a61c-bb43dc9cd2c5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:30.785406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3d7a9456-032a-43dc-af59-24424426288d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d0657d18ffb4372ad084150a9cfd121'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:30.786846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-160e0f9a-f678-48e7-a61c-bb43dc9cd2c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:30.790934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:30.793237 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3d7a9456-032a-43dc-af59-24424426288d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d0657d18ffb4372ad084150a9cfd121'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:30.793869 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3d7a9456-032a-43dc-af59-24424426288d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d0657d18ffb4372ad084150a9cfd121'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:30.795082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3d7a9456-032a-43dc-af59-24424426288d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d0657d18ffb4372ad084150a9cfd121'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:30.809089 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3d7a9456-032a-43dc-af59-24424426288d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d0657d18ffb4372ad084150a9cfd121'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:30.822090 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 943/1867] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:31:30 2022] DELETE /identity/v3/users/4d0657d18ffb4372ad084150a9cfd121 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:31:30.827747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af533dd1-ae37-49d8-8ca9-5d7b459adc40 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:30.828057 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af533dd1-ae37-49d8-8ca9-5d7b459adc40 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:30.828329 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af533dd1-ae37-49d8-8ca9-5d7b459adc40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:30.847961 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-af533dd1-ae37-49d8-8ca9-5d7b459adc40 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:30.849149 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-af533dd1-ae37-49d8-8ca9-5d7b459adc40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:30.856806 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e0f9a-f678-48e7-a61c-bb43dc9cd2c5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:30.908943 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 944/1868] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:31:30.917470 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d70ff-9700-484b-a389-467b823853fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:30.918319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d70ff-9700-484b-a389-467b823853fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:30.918604 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d70ff-9700-484b-a389-467b823853fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:30.918874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af533dd1-ae37-49d8-8ca9-5d7b459adc40 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:30.918969 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d70ff-9700-484b-a389-467b823853fa admin admin] PATH_INFO: `/v3/users/748fe541d54d49f896f735f3d7f350fd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:30.920038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d70ff-9700-484b-a389-467b823853fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=748fe541d54d49f896f735f3d7f350fd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:30.920477 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-af533dd1-ae37-49d8-8ca9-5d7b459adc40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:30.922032 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d70ff-9700-484b-a389-467b823853fa admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:30.947404 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:30.974239 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-381d70ff-9700-484b-a389-467b823853fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '748fe541d54d49f896f735f3d7f350fd'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:30.981185 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af533dd1-ae37-49d8-8ca9-5d7b459adc40 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:30.982122 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-381d70ff-9700-484b-a389-467b823853fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '748fe541d54d49f896f735f3d7f350fd'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:30.982863 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-381d70ff-9700-484b-a389-467b823853fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '748fe541d54d49f896f735f3d7f350fd'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:30.985047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9133c9-75f9-4850-ac90-7d2507ee819f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:30.985753 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9133c9-75f9-4850-ac90-7d2507ee819f service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:30.986096 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9133c9-75f9-4850-ac90-7d2507ee819f service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:30.986401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9133c9-75f9-4850-ac90-7d2507ee819f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:30.990631 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-381d70ff-9700-484b-a389-467b823853fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '748fe541d54d49f896f735f3d7f350fd'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:30.996561 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9133c9-75f9-4850-ac90-7d2507ee819f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:30.998057 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9133c9-75f9-4850-ac90-7d2507ee819f service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:31.004041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-381d70ff-9700-484b-a389-467b823853fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '748fe541d54d49f896f735f3d7f350fd'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:31.007629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 945/1869] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:31:31.010037 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 925/1870] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:31:30 2022] DELETE /identity/v3/users/748fe541d54d49f896f735f3d7f350fd => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:31:31.287476 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9133c9-75f9-4850-ac90-7d2507ee819f service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:31.288236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d70ff-9700-484b-a389-467b823853fa admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:31.289248 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ca9133c9-75f9-4850-ac90-7d2507ee819f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:31.290387 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-381d70ff-9700-484b-a389-467b823853fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:31.307433 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9133c9-75f9-4850-ac90-7d2507ee819f service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:31.405679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d70ff-9700-484b-a389-467b823853fa admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:31.410371 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424b257-b3a7-4606-87d4-1bf104cfd1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:31.410559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bc2e6-6174-45bb-aea6-95a5ba7b39f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:31.411447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424b257-b3a7-4606-87d4-1bf104cfd1d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:31.411797 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424b257-b3a7-4606-87d4-1bf104cfd1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:31.411906 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bc2e6-6174-45bb-aea6-95a5ba7b39f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:31.412189 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424b257-b3a7-4606-87d4-1bf104cfd1d7 admin admin] PATH_INFO: `/v3/projects/12178b7270c84a6aa14e43ef19eb7bad` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:31.412589 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bc2e6-6174-45bb-aea6-95a5ba7b39f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:31.413023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bc2e6-6174-45bb-aea6-95a5ba7b39f3 admin admin] PATH_INFO: `/v3/projects/a47490d8d86b4b0e92840d7479deb9ec` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:31.413445 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424b257-b3a7-4606-87d4-1bf104cfd1d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12178b7270c84a6aa14e43ef19eb7bad)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:31.415267 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455bc2e6-6174-45bb-aea6-95a5ba7b39f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a47490d8d86b4b0e92840d7479deb9ec)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:31.415523 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424b257-b3a7-4606-87d4-1bf104cfd1d7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:31.419007 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455bc2e6-6174-45bb-aea6-95a5ba7b39f3 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:31.447777 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:31.479734 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0424b257-b3a7-4606-87d4-1bf104cfd1d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12178b7270c84a6aa14e43ef19eb7bad', 'request_id': 'req-0424b257-b3a7-4606-87d4-1bf104cfd1d7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:31.488074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0424b257-b3a7-4606-87d4-1bf104cfd1d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12178b7270c84a6aa14e43ef19eb7bad', 'request_id': 'req-0424b257-b3a7-4606-87d4-1bf104cfd1d7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:31.497625 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0424b257-b3a7-4606-87d4-1bf104cfd1d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12178b7270c84a6aa14e43ef19eb7bad', 'request_id': 'req-0424b257-b3a7-4606-87d4-1bf104cfd1d7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:31.497850 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-455bc2e6-6174-45bb-aea6-95a5ba7b39f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a47490d8d86b4b0e92840d7479deb9ec', 'request_id': 'req-455bc2e6-6174-45bb-aea6-95a5ba7b39f3'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:31.511503 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0424b257-b3a7-4606-87d4-1bf104cfd1d7 admin admin] The token cache is being invalidate because project 12178b7270c84a6aa14e43ef19eb7bad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:31:31.512600 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0424b257-b3a7-4606-87d4-1bf104cfd1d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:31.513305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-455bc2e6-6174-45bb-aea6-95a5ba7b39f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a47490d8d86b4b0e92840d7479deb9ec', 'request_id': 'req-455bc2e6-6174-45bb-aea6-95a5ba7b39f3'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:31.513443 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0424b257-b3a7-4606-87d4-1bf104cfd1d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:31.515666 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 926/1871] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:31:31 2022] DELETE /identity/v3/projects/12178b7270c84a6aa14e43ef19eb7bad => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:31.521490 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb8caa-8454-47b4-8026-a6e4ff9b4b9a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:31.521649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-455bc2e6-6174-45bb-aea6-95a5ba7b39f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a47490d8d86b4b0e92840d7479deb9ec', 'request_id': 'req-455bc2e6-6174-45bb-aea6-95a5ba7b39f3'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:31.521746 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb8caa-8454-47b4-8026-a6e4ff9b4b9a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:31.521746 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb8caa-8454-47b4-8026-a6e4ff9b4b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:31.556104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-91fb8caa-8454-47b4-8026-a6e4ff9b4b9a None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:31.558459 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-91fb8caa-8454-47b4-8026-a6e4ff9b4b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:31.578220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-455bc2e6-6174-45bb-aea6-95a5ba7b39f3 admin admin] The token cache is being invalidate because project a47490d8d86b4b0e92840d7479deb9ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:31:31.578220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-455bc2e6-6174-45bb-aea6-95a5ba7b39f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:31.578220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-455bc2e6-6174-45bb-aea6-95a5ba7b39f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:31.578220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 946/1872] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:31:31 2022] DELETE /identity/v3/projects/a47490d8d86b4b0e92840d7479deb9ec => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:31.585253 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d594faa-6926-40d3-a2c1-60f7384fe49a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:31.585737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d594faa-6926-40d3-a2c1-60f7384fe49a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:31.586016 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d594faa-6926-40d3-a2c1-60f7384fe49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:31.612203 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:31.634685 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-8d594faa-6926-40d3-a2c1-60f7384fe49a None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:31.640168 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8d594faa-6926-40d3-a2c1-60f7384fe49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:31.697618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 927/1873] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:31:31.700104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569e954-e748-42a4-ad47-17fc5a978ea6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:31.707563 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569e954-e748-42a4-ad47-17fc5a978ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:31.707850 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569e954-e748-42a4-ad47-17fc5a978ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:31.750832 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d569e954-e748-42a4-ad47-17fc5a978ea6 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:31.758341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d569e954-e748-42a4-ad47-17fc5a978ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:31.763393 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 947/1874] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:31.773403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284effa8-06f5-45ad-bd5b-08097285132e None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:31.774135 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284effa8-06f5-45ad-bd5b-08097285132e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:31.775064 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284effa8-06f5-45ad-bd5b-08097285132e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:31.788599 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 928/1875] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:31:31.790895 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569e954-e748-42a4-ad47-17fc5a978ea6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:31.811620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569e954-e748-42a4-ad47-17fc5a978ea6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:31.825970 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f6b3dd-cdaa-44df-b8b8-5b6cdf7ea313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:31.826765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6b3dd-cdaa-44df-b8b8-5b6cdf7ea313 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:31.827401 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6b3dd-cdaa-44df-b8b8-5b6cdf7ea313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:31.827401 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6b3dd-cdaa-44df-b8b8-5b6cdf7ea313 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:31.828012 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f6b3dd-cdaa-44df-b8b8-5b6cdf7ea313 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:31:31.828312 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f6b3dd-cdaa-44df-b8b8-5b6cdf7ea313 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:31.830160 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f6b3dd-cdaa-44df-b8b8-5b6cdf7ea313 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:31.834295 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-284effa8-06f5-45ad-bd5b-08097285132e None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:31.837516 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-284effa8-06f5-45ad-bd5b-08097285132e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:31.853824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 929/1876] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:31:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:31:31.857414 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f6b3dd-cdaa-44df-b8b8-5b6cdf7ea313 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:31.862552 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 948/1877] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:31.865276 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284effa8-06f5-45ad-bd5b-08097285132e None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:31.873134 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f6b3dd-cdaa-44df-b8b8-5b6cdf7ea313 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:31.878876 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94fbe68-b185-45b4-a60d-433c959fc610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:31.879608 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fbe68-b185-45b4-a60d-433c959fc610 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:31.879996 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fbe68-b185-45b4-a60d-433c959fc610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:31.880387 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fbe68-b185-45b4-a60d-433c959fc610 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:31.881130 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94fbe68-b185-45b4-a60d-433c959fc610 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:31.883744 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94fbe68-b185-45b4-a60d-433c959fc610 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:31.885703 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284effa8-06f5-45ad-bd5b-08097285132e None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:31.892680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde48689-dcf2-4fc8-960c-12285dcbd5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:31.893751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde48689-dcf2-4fc8-960c-12285dcbd5b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:31.894171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde48689-dcf2-4fc8-960c-12285dcbd5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:31.895061 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde48689-dcf2-4fc8-960c-12285dcbd5b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:31.895235 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde48689-dcf2-4fc8-960c-12285dcbd5b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:31:31.895586 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde48689-dcf2-4fc8-960c-12285dcbd5b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:31.898723 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde48689-dcf2-4fc8-960c-12285dcbd5b3 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:31.917512 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 949/1878] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:31:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:31:31.922968 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde48689-dcf2-4fc8-960c-12285dcbd5b3 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:31.929934 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 930/1879] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:31:31 2022] POST /identity/v3/projects => generated 381 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:31.934085 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94fbe68-b185-45b4-a60d-433c959fc610 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:31.940833 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde48689-dcf2-4fc8-960c-12285dcbd5b3 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:32.016225 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94fbe68-b185-45b4-a60d-433c959fc610 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:32.022492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601cab28-889b-450d-b269-22bb2a9dde76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:32.023113 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cab28-889b-450d-b269-22bb2a9dde76 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:32.023433 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cab28-889b-450d-b269-22bb2a9dde76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:32.023717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cab28-889b-450d-b269-22bb2a9dde76 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:32.024301 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601cab28-889b-450d-b269-22bb2a9dde76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:32.026240 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601cab28-889b-450d-b269-22bb2a9dde76 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:32.028900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be32a76-1a91-444d-ad0b-d6b94e9ba190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:32.029507 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be32a76-1a91-444d-ad0b-d6b94e9ba190 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:32.029788 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be32a76-1a91-444d-ad0b-d6b94e9ba190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:32.030062 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be32a76-1a91-444d-ad0b-d6b94e9ba190 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:32.030736 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be32a76-1a91-444d-ad0b-d6b94e9ba190 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:32.032603 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be32a76-1a91-444d-ad0b-d6b94e9ba190 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:32.054163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 931/1880] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:31:31 2022] POST /identity/v3/users => generated 291 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:31:32.055242 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601cab28-889b-450d-b269-22bb2a9dde76 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:32.068168 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 950/1881] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:31:31 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:31:32.072433 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be32a76-1a91-444d-ad0b-d6b94e9ba190 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:32.074424 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601cab28-889b-450d-b269-22bb2a9dde76 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:32.086112 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:32.139097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f8258-447d-4cdc-82b7-beb05595fe95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:32.139739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f8258-447d-4cdc-82b7-beb05595fe95 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:32.140377 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f8258-447d-4cdc-82b7-beb05595fe95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:32.140661 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f8258-447d-4cdc-82b7-beb05595fe95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:32.141380 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f8258-447d-4cdc-82b7-beb05595fe95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:32.143033 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f8258-447d-4cdc-82b7-beb05595fe95 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:32.143825 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be32a76-1a91-444d-ad0b-d6b94e9ba190 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:32.148420 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da263c4e-9efd-40b5-8a4c-ad740f90bbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:32.149028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da263c4e-9efd-40b5-8a4c-ad740f90bbd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:32.149305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da263c4e-9efd-40b5-8a4c-ad740f90bbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:32.149580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da263c4e-9efd-40b5-8a4c-ad740f90bbd8 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:32.150227 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da263c4e-9efd-40b5-8a4c-ad740f90bbd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:32.152119 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da263c4e-9efd-40b5-8a4c-ad740f90bbd8 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:32.153547 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 932/1882] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:31:32.157588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f8258-447d-4cdc-82b7-beb05595fe95 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:32.168702 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f8258-447d-4cdc-82b7-beb05595fe95 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:32.173427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a351b-209c-4812-ba66-17e04c852780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:32.174042 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a351b-209c-4812-ba66-17e04c852780 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:32.174353 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a351b-209c-4812-ba66-17e04c852780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:32.177553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 951/1883] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:31:32 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:31:32.177677 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a351b-209c-4812-ba66-17e04c852780 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:32.177677 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a351b-209c-4812-ba66-17e04c852780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:32.177677 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a351b-209c-4812-ba66-17e04c852780 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:32.178536 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da263c4e-9efd-40b5-8a4c-ad740f90bbd8 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:32.188301 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 933/1884] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:31:32.192238 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a351b-209c-4812-ba66-17e04c852780 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:32.192400 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da263c4e-9efd-40b5-8a4c-ad740f90bbd8 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:32.196956 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2dd34d-df1f-403e-9dad-25cf46661d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:32.197547 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2dd34d-df1f-403e-9dad-25cf46661d48 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:32.197825 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2dd34d-df1f-403e-9dad-25cf46661d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:32.198099 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2dd34d-df1f-403e-9dad-25cf46661d48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:32.198787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2dd34d-df1f-403e-9dad-25cf46661d48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:32.198941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:32.200530 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2dd34d-df1f-403e-9dad-25cf46661d48 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:32.202899 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a351b-209c-4812-ba66-17e04c852780 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:32.208372 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cfa25-b5e0-431a-8b75-91a793733fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:32.209216 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cfa25-b5e0-431a-8b75-91a793733fd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:32.209502 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cfa25-b5e0-431a-8b75-91a793733fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:32.209777 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cfa25-b5e0-431a-8b75-91a793733fd9 admin admin] PATH_INFO: `/v3/projects/95595f65fa64428cab19797aa1d19a2d/users/868855c38a5c43f88acdc09c1c68145b/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:32.213857 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 952/1885] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:31:32.215104 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2dd34d-df1f-403e-9dad-25cf46661d48 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:32.224381 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12cfa25-b5e0-431a-8b75-91a793733fd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95595f65fa64428cab19797aa1d19a2d, user_id=868855c38a5c43f88acdc09c1c68145b, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:32.224714 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2dd34d-df1f-403e-9dad-25cf46661d48 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:32.227221 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12cfa25-b5e0-431a-8b75-91a793733fd9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:32.229597 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b330e75-7f51-4ac2-a304-ae8a5e87c69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:32.230226 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b330e75-7f51-4ac2-a304-ae8a5e87c69c admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:32.231031 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b330e75-7f51-4ac2-a304-ae8a5e87c69c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:32.231306 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b330e75-7f51-4ac2-a304-ae8a5e87c69c admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:32.231956 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b330e75-7f51-4ac2-a304-ae8a5e87c69c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:32.233555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b330e75-7f51-4ac2-a304-ae8a5e87c69c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:32.241384 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 934/1886] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:32 2022] PUT /identity/v3/projects/95595f65fa64428cab19797aa1d19a2d/users/868855c38a5c43f88acdc09c1c68145b/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:31:32.248149 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 953/1887] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:31:32.251193 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cfa25-b5e0-431a-8b75-91a793733fd9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:32.251753 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37782ac7-2104-46eb-8878-2ab59b4a27c6 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:32.252037 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37782ac7-2104-46eb-8878-2ab59b4a27c6 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:32.252310 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37782ac7-2104-46eb-8878-2ab59b4a27c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:32.279741 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-37782ac7-2104-46eb-8878-2ab59b4a27c6 None None] MFA Rules not processed for user `868855c38a5c43f88acdc09c1c68145b`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:32.280942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-37782ac7-2104-46eb-8878-2ab59b4a27c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:32.322944 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cfa25-b5e0-431a-8b75-91a793733fd9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:32.328521 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140b732-5269-4b69-8135-7b491592393f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:32.329761 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140b732-5269-4b69-8135-7b491592393f admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:32.330159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140b732-5269-4b69-8135-7b491592393f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:32.330802 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140b732-5269-4b69-8135-7b491592393f admin admin] PATH_INFO: `/v3/projects/d2a96a860fc84109afbda3e4b4e285d5/users/a2e56e99fe4942dfbd2b97e1b38ae1d9/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:32.347568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140b732-5269-4b69-8135-7b491592393f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2a96a860fc84109afbda3e4b4e285d5, user_id=a2e56e99fe4942dfbd2b97e1b38ae1d9, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:32.350589 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140b732-5269-4b69-8135-7b491592393f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:32.367473 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 935/1888] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:32 2022] PUT /identity/v3/projects/d2a96a860fc84109afbda3e4b4e285d5/users/a2e56e99fe4942dfbd2b97e1b38ae1d9/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:32.378157 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac289c6d-3c10-4c56-b479-e4981c9d32c5 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:32.378694 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac289c6d-3c10-4c56-b479-e4981c9d32c5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:32.379146 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac289c6d-3c10-4c56-b479-e4981c9d32c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:32.406426 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 954/1889] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:31:32.410793 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37782ac7-2104-46eb-8878-2ab59b4a27c6 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:32.419467 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ac289c6d-3c10-4c56-b479-e4981c9d32c5 None None] MFA Rules not processed for user `a2e56e99fe4942dfbd2b97e1b38ae1d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:32.422431 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ac289c6d-3c10-4c56-b479-e4981c9d32c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:32.427215 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37782ac7-2104-46eb-8878-2ab59b4a27c6 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:32.498725 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21440e-64a3-481b-9f57-f03e5c6d600d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:32.500132 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21440e-64a3-481b-9f57-f03e5c6d600d admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:32.500605 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21440e-64a3-481b-9f57-f03e5c6d600d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:32.501059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21440e-64a3-481b-9f57-f03e5c6d600d admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:32.501928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21440e-64a3-481b-9f57-f03e5c6d600d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:32.504252 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21440e-64a3-481b-9f57-f03e5c6d600d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:32.514336 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 936/1890] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:32.522426 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:32.529792 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac289c6d-3c10-4c56-b479-e4981c9d32c5 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:32.531947 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ac289c6d-3c10-4c56-b479-e4981c9d32c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:32.545680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 955/1891] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:31:32 2022] POST /identity/v3/projects => generated 379 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:31:32.546747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21440e-64a3-481b-9f57-f03e5c6d600d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:32.620347 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21440e-64a3-481b-9f57-f03e5c6d600d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:32.629616 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc1290a-1168-454c-9312-17eaa9c688a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:32.630642 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1290a-1168-454c-9312-17eaa9c688a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:32.631109 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1290a-1168-454c-9312-17eaa9c688a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:32.631562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1290a-1168-454c-9312-17eaa9c688a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:32.632392 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc1290a-1168-454c-9312-17eaa9c688a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:32.634670 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc1290a-1168-454c-9312-17eaa9c688a0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:32.635135 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac289c6d-3c10-4c56-b479-e4981c9d32c5 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:32.642184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:32.679160 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 956/1892] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:31:32 2022] POST /identity/v3/users => generated 296 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:32.682712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc1290a-1168-454c-9312-17eaa9c688a0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:32.711377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc1290a-1168-454c-9312-17eaa9c688a0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:32.718377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65922fc6-f64a-49cf-bc62-35a7ea797f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:32.719263 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65922fc6-f64a-49cf-bc62-35a7ea797f05 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:32.719897 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65922fc6-f64a-49cf-bc62-35a7ea797f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:32.720376 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65922fc6-f64a-49cf-bc62-35a7ea797f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:32.721252 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65922fc6-f64a-49cf-bc62-35a7ea797f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:32.723295 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65922fc6-f64a-49cf-bc62-35a7ea797f05 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:32.744503 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 957/1893] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:32.745720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65922fc6-f64a-49cf-bc62-35a7ea797f05 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:32.751675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eaa0ef-ca3d-4c77-b4c1-5433dabcf272 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:32.755897 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eaa0ef-ca3d-4c77-b4c1-5433dabcf272 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:32.756197 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eaa0ef-ca3d-4c77-b4c1-5433dabcf272 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:32.756505 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eaa0ef-ca3d-4c77-b4c1-5433dabcf272 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:32.759058 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65922fc6-f64a-49cf-bc62-35a7ea797f05 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:32.764350 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f84e5b1-1100-49ec-95f5-49257f9c0250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:32.765473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84e5b1-1100-49ec-95f5-49257f9c0250 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:32.765958 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84e5b1-1100-49ec-95f5-49257f9c0250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:32.766749 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84e5b1-1100-49ec-95f5-49257f9c0250 admin admin] PATH_INFO: `/v3/projects/71e259fdddc4480e9bfd48a768f7c2cb/users/63682f078544408790d36eb22a6c1653/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:32.791941 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f84e5b1-1100-49ec-95f5-49257f9c0250 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71e259fdddc4480e9bfd48a768f7c2cb, user_id=63682f078544408790d36eb22a6c1653, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:32.797565 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f84e5b1-1100-49ec-95f5-49257f9c0250 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:32.827110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 958/1894] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:32 2022] PUT /identity/v3/projects/71e259fdddc4480e9bfd48a768f7c2cb/users/63682f078544408790d36eb22a6c1653/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:32.832624 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f84e5b1-1100-49ec-95f5-49257f9c0250 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:32.932295 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eaa0ef-ca3d-4c77-b4c1-5433dabcf272 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:32.935562 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eaa0ef-ca3d-4c77-b4c1-5433dabcf272 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:32.961465 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 937/1895] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:32.981298 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f84e5b1-1100-49ec-95f5-49257f9c0250 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:32.987948 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e1379-71ea-4b1e-a6db-5b09e103992c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:32.988575 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e1379-71ea-4b1e-a6db-5b09e103992c admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:32.988856 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e1379-71ea-4b1e-a6db-5b09e103992c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:32.989131 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e1379-71ea-4b1e-a6db-5b09e103992c admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:32.989789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468e1379-71ea-4b1e-a6db-5b09e103992c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:32.991938 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468e1379-71ea-4b1e-a6db-5b09e103992c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:33.023590 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eaa0ef-ca3d-4c77-b4c1-5433dabcf272 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:33.024796 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 959/1896] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:33.068286 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eaa0ef-ca3d-4c77-b4c1-5433dabcf272 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:33.118067 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2202b-b201-4466-8e09-7b0e2db4e455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:33.121163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2202b-b201-4466-8e09-7b0e2db4e455 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:33.121453 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2202b-b201-4466-8e09-7b0e2db4e455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:33.121737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2202b-b201-4466-8e09-7b0e2db4e455 admin admin] PATH_INFO: `/v3/projects/71e259fdddc4480e9bfd48a768f7c2cb/users/63682f078544408790d36eb22a6c1653/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:33.128429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2202b-b201-4466-8e09-7b0e2db4e455 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71e259fdddc4480e9bfd48a768f7c2cb, user_id=63682f078544408790d36eb22a6c1653, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:33.132884 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2202b-b201-4466-8e09-7b0e2db4e455 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:33.164312 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 938/1897] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:33 2022] PUT /identity/v3/projects/71e259fdddc4480e9bfd48a768f7c2cb/users/63682f078544408790d36eb22a6c1653/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:31:33.176733 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92793407-b627-4da2-8f66-47159b1ca567 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:33.177038 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92793407-b627-4da2-8f66-47159b1ca567 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:33.177435 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92793407-b627-4da2-8f66-47159b1ca567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:33.241610 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-92793407-b627-4da2-8f66-47159b1ca567 None None] MFA Rules not processed for user `63682f078544408790d36eb22a6c1653`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:33.244206 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-92793407-b627-4da2-8f66-47159b1ca567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:33.315329 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:33.377970 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 960/1898] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:33 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:34.609512 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce605e9-765e-465e-9e3a-f423e008756b None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:34.610493 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce605e9-765e-465e-9e3a-f423e008756b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:34.610959 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce605e9-765e-465e-9e3a-f423e008756b None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:34.612517 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 939/1899] 10.222.0.58 () {54 vars in 921 bytes} [Thu Sep 1 16:31:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:31:34.630560 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546258f1-59c1-4e9d-9342-d603b834c6d0 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:34.631870 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546258f1-59c1-4e9d-9342-d603b834c6d0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:34.632402 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546258f1-59c1-4e9d-9342-d603b834c6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:34.677757 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-546258f1-59c1-4e9d-9342-d603b834c6d0 None None] MFA Rules not processed for user `1f0c99fdf8bd4331ad7e641ee38554c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:34.679286 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-546258f1-59c1-4e9d-9342-d603b834c6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:34.818433 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 961/1900] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Sep 1 16:31:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:34.888250 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce605e9-765e-465e-9e3a-f423e008756b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:34.890652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5ce605e9-765e-465e-9e3a-f423e008756b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:34.987590 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546258f1-59c1-4e9d-9342-d603b834c6d0 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:34.989039 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-546258f1-59c1-4e9d-9342-d603b834c6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:35.004695 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce605e9-765e-465e-9e3a-f423e008756b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:35.008533 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3bbae-199e-400e-9880-4adfc1c91d81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:35.009078 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3bbae-199e-400e-9880-4adfc1c91d81 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:35.009357 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3bbae-199e-400e-9880-4adfc1c91d81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:35.009627 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3bbae-199e-400e-9880-4adfc1c91d81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:35.011422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-abb3bbae-199e-400e-9880-4adfc1c91d81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:35.016531 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546258f1-59c1-4e9d-9342-d603b834c6d0 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:35.022716 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2cddc-a01d-4d49-9174-5e7f88284509 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:35.023331 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2cddc-a01d-4d49-9174-5e7f88284509 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:35.023611 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2cddc-a01d-4d49-9174-5e7f88284509 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:35.023884 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2cddc-a01d-4d49-9174-5e7f88284509 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:35.037869 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef2cddc-a01d-4d49-9174-5e7f88284509 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:35.039546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef2cddc-a01d-4d49-9174-5e7f88284509 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:35.066634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 962/1901] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:35.119613 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3bbae-199e-400e-9880-4adfc1c91d81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:35.123487 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3bbae-199e-400e-9880-4adfc1c91d81 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:35.140760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 940/1902] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:41.973456 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2cddc-a01d-4d49-9174-5e7f88284509 service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:41.983523 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2cddc-a01d-4d49-9174-5e7f88284509 service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:41.987177 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9138f0b-d942-4063-a3ff-9c3f7955eee6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:41.987786 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9138f0b-d942-4063-a3ff-9c3f7955eee6 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:41.988069 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9138f0b-d942-4063-a3ff-9c3f7955eee6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:41.988333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9138f0b-d942-4063-a3ff-9c3f7955eee6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:42.000883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9138f0b-d942-4063-a3ff-9c3f7955eee6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:42.002354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9138f0b-d942-4063-a3ff-9c3f7955eee6 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:42.016593 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 963/1903] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:42.037312 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3bbae-199e-400e-9880-4adfc1c91d81 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:42.039062 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-abb3bbae-199e-400e-9880-4adfc1c91d81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:42.046987 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:42.130939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3bbae-199e-400e-9880-4adfc1c91d81 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:42.135653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253f5aa-b5f6-43f6-9b3c-859e105f2aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:42.136319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253f5aa-b5f6-43f6-9b3c-859e105f2aae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:42.136600 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253f5aa-b5f6-43f6-9b3c-859e105f2aae admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:42.136878 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253f5aa-b5f6-43f6-9b3c-859e105f2aae admin admin] PATH_INFO: `/v3/users/0468f774ea7947099579aad6f50bddfb` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:42.137999 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b253f5aa-b5f6-43f6-9b3c-859e105f2aae admin admin] RBAC: Authorizing `identity:delete_user(user_id=0468f774ea7947099579aad6f50bddfb)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:42.139966 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b253f5aa-b5f6-43f6-9b3c-859e105f2aae admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:42.189996 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b253f5aa-b5f6-43f6-9b3c-859e105f2aae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0468f774ea7947099579aad6f50bddfb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:42.197077 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b253f5aa-b5f6-43f6-9b3c-859e105f2aae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0468f774ea7947099579aad6f50bddfb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:42.197756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b253f5aa-b5f6-43f6-9b3c-859e105f2aae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0468f774ea7947099579aad6f50bddfb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:42.206794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b253f5aa-b5f6-43f6-9b3c-859e105f2aae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0468f774ea7947099579aad6f50bddfb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:42.221274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b253f5aa-b5f6-43f6-9b3c-859e105f2aae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0468f774ea7947099579aad6f50bddfb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:42.227848 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 941/1904] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:31:42 2022] DELETE /identity/v3/users/0468f774ea7947099579aad6f50bddfb => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:42.233062 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c65bf-4b3f-4841-bf39-7c89a57ba106 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:42.233347 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c65bf-4b3f-4841-bf39-7c89a57ba106 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:42.233620 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c65bf-4b3f-4841-bf39-7c89a57ba106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:42.254389 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-8f3c65bf-4b3f-4841-bf39-7c89a57ba106 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:42.255506 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8f3c65bf-4b3f-4841-bf39-7c89a57ba106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:42.327484 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 964/1905] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:42.337318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253f5aa-b5f6-43f6-9b3c-859e105f2aae admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:42.338730 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b253f5aa-b5f6-43f6-9b3c-859e105f2aae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:42.358524 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3c65bf-4b3f-4841-bf39-7c89a57ba106 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:42.359957 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8f3c65bf-4b3f-4841-bf39-7c89a57ba106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:42.396797 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253f5aa-b5f6-43f6-9b3c-859e105f2aae admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:42.399195 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:42.401329 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6593e-ef56-4cbf-8752-f4bf2cb58ce0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:42.402506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6593e-ef56-4cbf-8752-f4bf2cb58ce0 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:42.402793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6593e-ef56-4cbf-8752-f4bf2cb58ce0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:42.403059 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6593e-ef56-4cbf-8752-f4bf2cb58ce0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:42.412349 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c6593e-ef56-4cbf-8752-f4bf2cb58ce0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:42.414021 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c6593e-ef56-4cbf-8752-f4bf2cb58ce0 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:42.423866 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 942/1906] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:42.426232 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3c65bf-4b3f-4841-bf39-7c89a57ba106 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:42.430739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf56c26-51e1-4b0a-9b9a-7d5f9c3b9af7 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63682f078544408790d36eb22a6c1653', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e259fdddc4480e9bfd48a768f7c2cb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:42.431337 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf56c26-51e1-4b0a-9b9a-7d5f9c3b9af7 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:42.431620 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf56c26-51e1-4b0a-9b9a-7d5f9c3b9af7 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:42.431893 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf56c26-51e1-4b0a-9b9a-7d5f9c3b9af7 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:42.432544 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf56c26-51e1-4b0a-9b9a-7d5f9c3b9af7 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:31:42.432848 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf56c26-51e1-4b0a-9b9a-7d5f9c3b9af7 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:42.434612 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf56c26-51e1-4b0a-9b9a-7d5f9c3b9af7 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:42.446331 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 965/1907] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:31:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:31:42.450940 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6593e-ef56-4cbf-8752-f4bf2cb58ce0 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:42.461160 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6593e-ef56-4cbf-8752-f4bf2cb58ce0 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:42.466317 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51350fc4-b9af-4544-86ba-58efd905db0a tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63682f078544408790d36eb22a6c1653', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e259fdddc4480e9bfd48a768f7c2cb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:42.466922 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51350fc4-b9af-4544-86ba-58efd905db0a tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:42.467211 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51350fc4-b9af-4544-86ba-58efd905db0a tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:42.467490 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51350fc4-b9af-4544-86ba-58efd905db0a tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:42.468139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51350fc4-b9af-4544-86ba-58efd905db0a tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:42.470047 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51350fc4-b9af-4544-86ba-58efd905db0a tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:42.502284 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 943/1908] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:31:42 2022] POST /identity/v3/projects => generated 354 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:42.506535 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf56c26-51e1-4b0a-9b9a-7d5f9c3b9af7 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:42.523081 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf56c26-51e1-4b0a-9b9a-7d5f9c3b9af7 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:42.595973 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de45e65-b4dd-404a-8aa6-72bdc9f05bf0 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63682f078544408790d36eb22a6c1653', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e259fdddc4480e9bfd48a768f7c2cb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:42.596688 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de45e65-b4dd-404a-8aa6-72bdc9f05bf0 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:42.597067 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de45e65-b4dd-404a-8aa6-72bdc9f05bf0 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:42.597459 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de45e65-b4dd-404a-8aa6-72bdc9f05bf0 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:42.598240 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de45e65-b4dd-404a-8aa6-72bdc9f05bf0 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:31:42.598656 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de45e65-b4dd-404a-8aa6-72bdc9f05bf0 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:42.600741 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de45e65-b4dd-404a-8aa6-72bdc9f05bf0 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:42.618213 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 966/1909] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:31:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:31:42.662734 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51350fc4-b9af-4544-86ba-58efd905db0a tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:42.672951 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:42.679219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51350fc4-b9af-4544-86ba-58efd905db0a tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:42.688787 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be4673-f956-4b5b-858b-290cdf97c430 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63682f078544408790d36eb22a6c1653', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e259fdddc4480e9bfd48a768f7c2cb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:42.689505 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be4673-f956-4b5b-858b-290cdf97c430 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:42.689799 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be4673-f956-4b5b-858b-290cdf97c430 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:42.690080 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be4673-f956-4b5b-858b-290cdf97c430 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] PATH_INFO: `/v3/projects/82a7e66ff18f4f53808b67cf23c68753` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:42.691412 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be4673-f956-4b5b-858b-290cdf97c430 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] RBAC: Authorizing `identity:delete_project(project_id=82a7e66ff18f4f53808b67cf23c68753)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:42.695960 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be4673-f956-4b5b-858b-290cdf97c430 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:42.794950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de45e65-b4dd-404a-8aa6-72bdc9f05bf0 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:42.794950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0de45e65-b4dd-404a-8aa6-72bdc9f05bf0 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:42.815158 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-34be4673-f956-4b5b-858b-290cdf97c430 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82a7e66ff18f4f53808b67cf23c68753', 'request_id': 'req-34be4673-f956-4b5b-858b-290cdf97c430'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:42.834262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-34be4673-f956-4b5b-858b-290cdf97c430 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82a7e66ff18f4f53808b67cf23c68753', 'request_id': 'req-34be4673-f956-4b5b-858b-290cdf97c430'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:42.845335 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-34be4673-f956-4b5b-858b-290cdf97c430 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82a7e66ff18f4f53808b67cf23c68753', 'request_id': 'req-34be4673-f956-4b5b-858b-290cdf97c430'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:42.863289 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-34be4673-f956-4b5b-858b-290cdf97c430 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] The token cache is being invalidate because project 82a7e66ff18f4f53808b67cf23c68753 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:31:42.866140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-34be4673-f956-4b5b-858b-290cdf97c430 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:42.866140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-34be4673-f956-4b5b-858b-290cdf97c430 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:42.871823 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 944/1910] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:31:42 2022] DELETE /identity/v3/projects/82a7e66ff18f4f53808b67cf23c68753 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:42.913868 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de45e65-b4dd-404a-8aa6-72bdc9f05bf0 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:43.011553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92894eeb-14f7-45cd-8357-a3698ce0376e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:43.012192 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92894eeb-14f7-45cd-8357-a3698ce0376e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:43.012476 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92894eeb-14f7-45cd-8357-a3698ce0376e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:43.012889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92894eeb-14f7-45cd-8357-a3698ce0376e admin admin] PATH_INFO: `/v3/projects/f9d10651ad0f4bd5a57c6df3c28878f2` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:43.014131 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92894eeb-14f7-45cd-8357-a3698ce0376e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9d10651ad0f4bd5a57c6df3c28878f2)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:43.016376 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92894eeb-14f7-45cd-8357-a3698ce0376e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:43.058732 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:43.124272 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-92894eeb-14f7-45cd-8357-a3698ce0376e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9d10651ad0f4bd5a57c6df3c28878f2', 'request_id': 'req-92894eeb-14f7-45cd-8357-a3698ce0376e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:43.133395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-92894eeb-14f7-45cd-8357-a3698ce0376e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9d10651ad0f4bd5a57c6df3c28878f2', 'request_id': 'req-92894eeb-14f7-45cd-8357-a3698ce0376e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:43.142858 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-92894eeb-14f7-45cd-8357-a3698ce0376e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9d10651ad0f4bd5a57c6df3c28878f2', 'request_id': 'req-92894eeb-14f7-45cd-8357-a3698ce0376e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:43.188236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-92894eeb-14f7-45cd-8357-a3698ce0376e admin admin] The token cache is being invalidate because project f9d10651ad0f4bd5a57c6df3c28878f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:31:43.189961 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-92894eeb-14f7-45cd-8357-a3698ce0376e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:43.193996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-92894eeb-14f7-45cd-8357-a3698ce0376e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:43.203979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 967/1911] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:31:42 2022] DELETE /identity/v3/projects/f9d10651ad0f4bd5a57c6df3c28878f2 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:43.204731 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be4673-f956-4b5b-858b-290cdf97c430 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:43.206917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-34be4673-f956-4b5b-858b-290cdf97c430 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:43.237025 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be4673-f956-4b5b-858b-290cdf97c430 tempest-VolumeQuotaClassesTest-807561750 tempest-VolumeQuotaClassesTest-807561750-project-admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:43.241827 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92894eeb-14f7-45cd-8357-a3698ce0376e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:43.243412 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-92894eeb-14f7-45cd-8357-a3698ce0376e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:43.371291 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7b432-237c-42fd-8f3e-013e114bfbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:43.371929 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7b432-237c-42fd-8f3e-013e114bfbc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:43.372208 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7b432-237c-42fd-8f3e-013e114bfbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:43.372478 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7b432-237c-42fd-8f3e-013e114bfbc3 admin admin] PATH_INFO: `/v3/users/854b04e7b10d4d739f50454c1cd38156` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:43.373600 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed7b432-237c-42fd-8f3e-013e114bfbc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=854b04e7b10d4d739f50454c1cd38156)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:43.375535 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed7b432-237c-42fd-8f3e-013e114bfbc3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:43.387749 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92894eeb-14f7-45cd-8357-a3698ce0376e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:43.394068 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8ca08-240d-4cef-ab7e-d81d6ed0ee12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:43.394761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8ca08-240d-4cef-ab7e-d81d6ed0ee12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:43.395039 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8ca08-240d-4cef-ab7e-d81d6ed0ee12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:43.395307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8ca08-240d-4cef-ab7e-d81d6ed0ee12 admin admin] PATH_INFO: `/v3/users/868855c38a5c43f88acdc09c1c68145b` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:43.396513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8ca08-240d-4cef-ab7e-d81d6ed0ee12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=868855c38a5c43f88acdc09c1c68145b)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:43.398467 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8ca08-240d-4cef-ab7e-d81d6ed0ee12 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:43.432556 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-aed7b432-237c-42fd-8f3e-013e114bfbc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '854b04e7b10d4d739f50454c1cd38156'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:43.443812 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-aed7b432-237c-42fd-8f3e-013e114bfbc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '854b04e7b10d4d739f50454c1cd38156'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:43.444439 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-aed7b432-237c-42fd-8f3e-013e114bfbc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '854b04e7b10d4d739f50454c1cd38156'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:43.454006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-aed7b432-237c-42fd-8f3e-013e114bfbc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '854b04e7b10d4d739f50454c1cd38156'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:43.459742 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-aac8ca08-240d-4cef-ab7e-d81d6ed0ee12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '868855c38a5c43f88acdc09c1c68145b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:43.468934 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-aed7b432-237c-42fd-8f3e-013e114bfbc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '854b04e7b10d4d739f50454c1cd38156'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:43.469208 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-aac8ca08-240d-4cef-ab7e-d81d6ed0ee12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '868855c38a5c43f88acdc09c1c68145b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:43.469937 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-aac8ca08-240d-4cef-ab7e-d81d6ed0ee12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '868855c38a5c43f88acdc09c1c68145b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:43.471171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-aac8ca08-240d-4cef-ab7e-d81d6ed0ee12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '868855c38a5c43f88acdc09c1c68145b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:43.476835 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 945/1912] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:31:43 2022] DELETE /identity/v3/users/854b04e7b10d4d739f50454c1cd38156 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:43.492031 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-aac8ca08-240d-4cef-ab7e-d81d6ed0ee12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '868855c38a5c43f88acdc09c1c68145b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:43.514279 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 968/1913] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:31:43 2022] DELETE /identity/v3/users/868855c38a5c43f88acdc09c1c68145b => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:43.519178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d3877-a656-4de3-9656-dc289d1ebd67 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:43.519633 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d3877-a656-4de3-9656-dc289d1ebd67 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:43.520281 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d3877-a656-4de3-9656-dc289d1ebd67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:43.537365 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:43.545044 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-239d3877-a656-4de3-9656-dc289d1ebd67 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:43.547749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-239d3877-a656-4de3-9656-dc289d1ebd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:43.630137 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 946/1914] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:43.640137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8ca08-240d-4cef-ab7e-d81d6ed0ee12 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:43.641717 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-aac8ca08-240d-4cef-ab7e-d81d6ed0ee12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:43.674492 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:43.717771 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8ca08-240d-4cef-ab7e-d81d6ed0ee12 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:43.721591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cfdd64-32c1-4b9a-8bfa-6f3cead449e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:43.722219 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cfdd64-32c1-4b9a-8bfa-6f3cead449e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:43.722505 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cfdd64-32c1-4b9a-8bfa-6f3cead449e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:43.722777 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cfdd64-32c1-4b9a-8bfa-6f3cead449e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:43.733437 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cfdd64-32c1-4b9a-8bfa-6f3cead449e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:43.734965 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cfdd64-32c1-4b9a-8bfa-6f3cead449e5 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:43.746422 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 969/1915] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:44.110628 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d3877-a656-4de3-9656-dc289d1ebd67 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:44.112248 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-239d3877-a656-4de3-9656-dc289d1ebd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:44.124862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d3877-a656-4de3-9656-dc289d1ebd67 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:44.129703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8030b8a6-4d10-41dc-b0b1-f23e15f5b19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:44.130389 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030b8a6-4d10-41dc-b0b1-f23e15f5b19f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:44.130650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030b8a6-4d10-41dc-b0b1-f23e15f5b19f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:44.130919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030b8a6-4d10-41dc-b0b1-f23e15f5b19f admin admin] PATH_INFO: `/v3/projects/63deaa5cbb0c41d29f986991c9e04020` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:44.132042 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8030b8a6-4d10-41dc-b0b1-f23e15f5b19f admin admin] RBAC: Authorizing `identity:delete_project(project_id=63deaa5cbb0c41d29f986991c9e04020)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:44.133936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8030b8a6-4d10-41dc-b0b1-f23e15f5b19f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:44.184097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8030b8a6-4d10-41dc-b0b1-f23e15f5b19f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63deaa5cbb0c41d29f986991c9e04020', 'request_id': 'req-8030b8a6-4d10-41dc-b0b1-f23e15f5b19f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:44.190734 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8030b8a6-4d10-41dc-b0b1-f23e15f5b19f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63deaa5cbb0c41d29f986991c9e04020', 'request_id': 'req-8030b8a6-4d10-41dc-b0b1-f23e15f5b19f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:44.200422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8030b8a6-4d10-41dc-b0b1-f23e15f5b19f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63deaa5cbb0c41d29f986991c9e04020', 'request_id': 'req-8030b8a6-4d10-41dc-b0b1-f23e15f5b19f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:44.217509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8030b8a6-4d10-41dc-b0b1-f23e15f5b19f admin admin] The token cache is being invalidate because project 63deaa5cbb0c41d29f986991c9e04020 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:31:44.218073 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8030b8a6-4d10-41dc-b0b1-f23e15f5b19f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:44.218779 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8030b8a6-4d10-41dc-b0b1-f23e15f5b19f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:44.220926 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 947/1916] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:31:44 2022] DELETE /identity/v3/projects/63deaa5cbb0c41d29f986991c9e04020 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:44.224730 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cfdd64-32c1-4b9a-8bfa-6f3cead449e5 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:44.244092 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cfdd64-32c1-4b9a-8bfa-6f3cead449e5 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:44.357513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01e941-6104-4e61-afc9-594ec975c7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:44.358834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01e941-6104-4e61-afc9-594ec975c7ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:44.359567 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01e941-6104-4e61-afc9-594ec975c7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:44.360065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01e941-6104-4e61-afc9-594ec975c7ec admin admin] PATH_INFO: `/v3/users/97b1e198cbc1435e8fab9fb9f9714391` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:44.361523 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b01e941-6104-4e61-afc9-594ec975c7ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=97b1e198cbc1435e8fab9fb9f9714391)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:44.366391 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b01e941-6104-4e61-afc9-594ec975c7ec admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:44.393520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8030b8a6-4d10-41dc-b0b1-f23e15f5b19f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:44.395012 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8030b8a6-4d10-41dc-b0b1-f23e15f5b19f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:44.407848 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8030b8a6-4d10-41dc-b0b1-f23e15f5b19f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:44.413118 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52038d8-f0d4-45ec-a437-d45ab3d979b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:44.414154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52038d8-f0d4-45ec-a437-d45ab3d979b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:44.414154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52038d8-f0d4-45ec-a437-d45ab3d979b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:44.414471 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52038d8-f0d4-45ec-a437-d45ab3d979b5 admin admin] PATH_INFO: `/v3/users/a2e56e99fe4942dfbd2b97e1b38ae1d9` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:44.415650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52038d8-f0d4-45ec-a437-d45ab3d979b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2e56e99fe4942dfbd2b97e1b38ae1d9)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:44.418102 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52038d8-f0d4-45ec-a437-d45ab3d979b5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:44.463195 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1b01e941-6104-4e61-afc9-594ec975c7ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97b1e198cbc1435e8fab9fb9f9714391'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:44.477922 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1b01e941-6104-4e61-afc9-594ec975c7ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97b1e198cbc1435e8fab9fb9f9714391'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:44.478894 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1b01e941-6104-4e61-afc9-594ec975c7ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97b1e198cbc1435e8fab9fb9f9714391'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:44.480386 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1b01e941-6104-4e61-afc9-594ec975c7ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97b1e198cbc1435e8fab9fb9f9714391'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:44.485467 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b52038d8-f0d4-45ec-a437-d45ab3d979b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2e56e99fe4942dfbd2b97e1b38ae1d9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:44.494094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b52038d8-f0d4-45ec-a437-d45ab3d979b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2e56e99fe4942dfbd2b97e1b38ae1d9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:44.494959 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b52038d8-f0d4-45ec-a437-d45ab3d979b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2e56e99fe4942dfbd2b97e1b38ae1d9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:44.506839 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b52038d8-f0d4-45ec-a437-d45ab3d979b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2e56e99fe4942dfbd2b97e1b38ae1d9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:44.508214 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1b01e941-6104-4e61-afc9-594ec975c7ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97b1e198cbc1435e8fab9fb9f9714391'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:44.523246 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 970/1917] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:31:44 2022] DELETE /identity/v3/users/97b1e198cbc1435e8fab9fb9f9714391 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:31:44.524096 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01e941-6104-4e61-afc9-594ec975c7ec admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:44.525446 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1b01e941-6104-4e61-afc9-594ec975c7ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:44.529038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b52038d8-f0d4-45ec-a437-d45ab3d979b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2e56e99fe4942dfbd2b97e1b38ae1d9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:44.537372 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 948/1918] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:31:44 2022] DELETE /identity/v3/users/a2e56e99fe4942dfbd2b97e1b38ae1d9 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:31:44.542077 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d45e9-d4ea-4ba0-9b66-8f8b9c452ac3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:44.543081 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d45e9-d4ea-4ba0-9b66-8f8b9c452ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:44.543542 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d45e9-d4ea-4ba0-9b66-8f8b9c452ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:44.554470 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:44.577935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6c5d45e9-d4ea-4ba0-9b66-8f8b9c452ac3 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:44.579288 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6c5d45e9-d4ea-4ba0-9b66-8f8b9c452ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:44.590416 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:44.631083 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01e941-6104-4e61-afc9-594ec975c7ec admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:44.653569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 949/1919] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:44.669917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d45e9-d4ea-4ba0-9b66-8f8b9c452ac3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:44.672456 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6c5d45e9-d4ea-4ba0-9b66-8f8b9c452ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:44.719439 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b348717-1554-4571-9d0b-2800a8951dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:44.720286 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b348717-1554-4571-9d0b-2800a8951dcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:44.720630 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b348717-1554-4571-9d0b-2800a8951dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:44.720969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b348717-1554-4571-9d0b-2800a8951dcd admin admin] PATH_INFO: `/v3/projects/95595f65fa64428cab19797aa1d19a2d` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:44.722305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b348717-1554-4571-9d0b-2800a8951dcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=95595f65fa64428cab19797aa1d19a2d)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:44.724624 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b348717-1554-4571-9d0b-2800a8951dcd admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:44.792098 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5b348717-1554-4571-9d0b-2800a8951dcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95595f65fa64428cab19797aa1d19a2d', 'request_id': 'req-5b348717-1554-4571-9d0b-2800a8951dcd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:44.797505 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d45e9-d4ea-4ba0-9b66-8f8b9c452ac3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:44.801496 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5b348717-1554-4571-9d0b-2800a8951dcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95595f65fa64428cab19797aa1d19a2d', 'request_id': 'req-5b348717-1554-4571-9d0b-2800a8951dcd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:44.809975 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5b348717-1554-4571-9d0b-2800a8951dcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95595f65fa64428cab19797aa1d19a2d', 'request_id': 'req-5b348717-1554-4571-9d0b-2800a8951dcd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:44.830814 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5b348717-1554-4571-9d0b-2800a8951dcd admin admin] The token cache is being invalidate because project 95595f65fa64428cab19797aa1d19a2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:31:44.831069 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5b348717-1554-4571-9d0b-2800a8951dcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:44.832508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5b348717-1554-4571-9d0b-2800a8951dcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:44.837142 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 971/1920] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:31:44 2022] DELETE /identity/v3/projects/95595f65fa64428cab19797aa1d19a2d => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:44.838377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b348717-1554-4571-9d0b-2800a8951dcd admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:44.840090 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5b348717-1554-4571-9d0b-2800a8951dcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:44.877684 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd5ded-bc01-4fd5-b42a-47ea8b783288 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:44.878980 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd5ded-bc01-4fd5-b42a-47ea8b783288 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:44.879758 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd5ded-bc01-4fd5-b42a-47ea8b783288 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:44.880243 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd5ded-bc01-4fd5-b42a-47ea8b783288 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:44.882760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c0cd5ded-bc01-4fd5-b42a-47ea8b783288 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:44.921532 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b348717-1554-4571-9d0b-2800a8951dcd admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:44.927213 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032c802-9076-41b3-a17e-c9c71c12161d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:44.928146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032c802-9076-41b3-a17e-c9c71c12161d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:44.928493 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032c802-9076-41b3-a17e-c9c71c12161d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:44.928834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032c802-9076-41b3-a17e-c9c71c12161d admin admin] PATH_INFO: `/v3/users/63682f078544408790d36eb22a6c1653` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:44.930310 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d032c802-9076-41b3-a17e-c9c71c12161d admin admin] RBAC: Authorizing `identity:delete_user(user_id=63682f078544408790d36eb22a6c1653)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:44.932302 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d032c802-9076-41b3-a17e-c9c71c12161d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:44.962546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:44.986737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd5ded-bc01-4fd5-b42a-47ea8b783288 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:44.988997 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd5ded-bc01-4fd5-b42a-47ea8b783288 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:45.007552 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 950/1921] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:45.014581 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d032c802-9076-41b3-a17e-c9c71c12161d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63682f078544408790d36eb22a6c1653'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:45.023138 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d032c802-9076-41b3-a17e-c9c71c12161d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63682f078544408790d36eb22a6c1653'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:45.023895 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d032c802-9076-41b3-a17e-c9c71c12161d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63682f078544408790d36eb22a6c1653'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:45.024521 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d032c802-9076-41b3-a17e-c9c71c12161d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63682f078544408790d36eb22a6c1653'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:45.042872 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d032c802-9076-41b3-a17e-c9c71c12161d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63682f078544408790d36eb22a6c1653'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:45.062696 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 972/1922] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:31:44 2022] DELETE /identity/v3/users/63682f078544408790d36eb22a6c1653 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:45.127323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd5ded-bc01-4fd5-b42a-47ea8b783288 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:45.128844 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c0cd5ded-bc01-4fd5-b42a-47ea8b783288 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:45.144066 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd5ded-bc01-4fd5-b42a-47ea8b783288 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:45.151140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:45.236494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984eb145-8808-4496-9282-69164dcf68c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:45.237313 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984eb145-8808-4496-9282-69164dcf68c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:45.237779 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984eb145-8808-4496-9282-69164dcf68c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:45.238067 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984eb145-8808-4496-9282-69164dcf68c1 admin admin] PATH_INFO: `/v3/projects/3e2c0f4d09d841f2b8550e611461ad6d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:45.239240 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984eb145-8808-4496-9282-69164dcf68c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e2c0f4d09d841f2b8550e611461ad6d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:45.241501 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984eb145-8808-4496-9282-69164dcf68c1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:45.318361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-984eb145-8808-4496-9282-69164dcf68c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e2c0f4d09d841f2b8550e611461ad6d', 'request_id': 'req-984eb145-8808-4496-9282-69164dcf68c1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:45.326028 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-984eb145-8808-4496-9282-69164dcf68c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e2c0f4d09d841f2b8550e611461ad6d', 'request_id': 'req-984eb145-8808-4496-9282-69164dcf68c1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:45.336918 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-984eb145-8808-4496-9282-69164dcf68c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e2c0f4d09d841f2b8550e611461ad6d', 'request_id': 'req-984eb145-8808-4496-9282-69164dcf68c1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:45.352022 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-984eb145-8808-4496-9282-69164dcf68c1 admin admin] The token cache is being invalidate because project 3e2c0f4d09d841f2b8550e611461ad6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:31:45.352404 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-984eb145-8808-4496-9282-69164dcf68c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:45.353194 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-984eb145-8808-4496-9282-69164dcf68c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:45.360178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 951/1923] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:31:45 2022] DELETE /identity/v3/projects/3e2c0f4d09d841f2b8550e611461ad6d => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:45.361894 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf54f4-4d14-492f-94e7-9b8f0019d887 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:45.362368 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf54f4-4d14-492f-94e7-9b8f0019d887 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:45.362506 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf54f4-4d14-492f-94e7-9b8f0019d887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:45.383174 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-87cf54f4-4d14-492f-94e7-9b8f0019d887 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:45.384399 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-87cf54f4-4d14-492f-94e7-9b8f0019d887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:45.463711 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 973/1924] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:45.465805 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d71db7-c426-467e-af98-5096d2d8d38f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:45.466098 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d71db7-c426-467e-af98-5096d2d8d38f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:45.466406 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d71db7-c426-467e-af98-5096d2d8d38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:45.487502 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-22d71db7-c426-467e-af98-5096d2d8d38f None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:45.488743 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-22d71db7-c426-467e-af98-5096d2d8d38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:45.497993 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 952/1925] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:45.503007 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf54f4-4d14-492f-94e7-9b8f0019d887 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:45.519222 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf54f4-4d14-492f-94e7-9b8f0019d887 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:45.525239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a39742-930d-4bf8-93af-ad46ba2a1739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:45.525854 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a39742-930d-4bf8-93af-ad46ba2a1739 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:45.526139 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a39742-930d-4bf8-93af-ad46ba2a1739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:45.526446 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a39742-930d-4bf8-93af-ad46ba2a1739 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:45.527665 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a39742-930d-4bf8-93af-ad46ba2a1739 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:31:45.527976 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a39742-930d-4bf8-93af-ad46ba2a1739 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:45.529593 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a39742-930d-4bf8-93af-ad46ba2a1739 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:45.545856 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 974/1926] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:31:45.550908 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d71db7-c426-467e-af98-5096d2d8d38f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:45.562035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d71db7-c426-467e-af98-5096d2d8d38f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:45.567642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606bc49-9187-4379-973b-378f82f044f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:45.568270 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606bc49-9187-4379-973b-378f82f044f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:45.568557 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606bc49-9187-4379-973b-378f82f044f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:45.568852 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606bc49-9187-4379-973b-378f82f044f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:45.569482 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4606bc49-9187-4379-973b-378f82f044f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:45.573925 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4606bc49-9187-4379-973b-378f82f044f2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:45.597162 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a39742-930d-4bf8-93af-ad46ba2a1739 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:45.599103 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a2a39742-930d-4bf8-93af-ad46ba2a1739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:45.605645 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 953/1927] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:31:45 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:45.610017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606bc49-9187-4379-973b-378f82f044f2 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:45.617701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a39742-930d-4bf8-93af-ad46ba2a1739 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:45.682964 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ade5fc-545e-4978-aeab-69dce646f8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:45.688633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ade5fc-545e-4978-aeab-69dce646f8c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:45.688920 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ade5fc-545e-4978-aeab-69dce646f8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:45.689214 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ade5fc-545e-4978-aeab-69dce646f8c7 admin admin] PATH_INFO: `/v3/projects/d2a96a860fc84109afbda3e4b4e285d5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:45.690686 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ade5fc-545e-4978-aeab-69dce646f8c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2a96a860fc84109afbda3e4b4e285d5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:45.692606 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ade5fc-545e-4978-aeab-69dce646f8c7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:45.753280 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606bc49-9187-4379-973b-378f82f044f2 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:45.762094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:45.763009 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4de69-f0e7-42ba-813b-6751b9478aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:45.763773 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4de69-f0e7-42ba-813b-6751b9478aad admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:45.763915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4de69-f0e7-42ba-813b-6751b9478aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:45.764197 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4de69-f0e7-42ba-813b-6751b9478aad admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:45.765097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4de69-f0e7-42ba-813b-6751b9478aad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:45.769923 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4de69-f0e7-42ba-813b-6751b9478aad admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:45.792727 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:45.795613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c1ade5fc-545e-4978-aeab-69dce646f8c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2a96a860fc84109afbda3e4b4e285d5', 'request_id': 'req-c1ade5fc-545e-4978-aeab-69dce646f8c7'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:45.803383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c1ade5fc-545e-4978-aeab-69dce646f8c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2a96a860fc84109afbda3e4b4e285d5', 'request_id': 'req-c1ade5fc-545e-4978-aeab-69dce646f8c7'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:45.805620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 954/1928] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:31:45 2022] POST /identity/v3/users => generated 294 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:31:45.806540 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4de69-f0e7-42ba-813b-6751b9478aad admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:45.809990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-12c4de69-f0e7-42ba-813b-6751b9478aad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:45.813500 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c1ade5fc-545e-4978-aeab-69dce646f8c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2a96a860fc84109afbda3e4b4e285d5', 'request_id': 'req-c1ade5fc-545e-4978-aeab-69dce646f8c7'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:45.824833 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4de69-f0e7-42ba-813b-6751b9478aad admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:45.830783 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c1ade5fc-545e-4978-aeab-69dce646f8c7 admin admin] The token cache is being invalidate because project d2a96a860fc84109afbda3e4b4e285d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:31:45.830953 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c1ade5fc-545e-4978-aeab-69dce646f8c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:45.831612 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c1ade5fc-545e-4978-aeab-69dce646f8c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:45.833258 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 975/1929] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:31:45 2022] DELETE /identity/v3/projects/d2a96a860fc84109afbda3e4b4e285d5 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:45.834027 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ade5fc-545e-4978-aeab-69dce646f8c7 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:45.835408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c1ade5fc-545e-4978-aeab-69dce646f8c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:45.897831 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a944b83-f30e-4b95-9532-6617bc7d62db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:45.899149 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a944b83-f30e-4b95-9532-6617bc7d62db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:45.899508 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a944b83-f30e-4b95-9532-6617bc7d62db admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:45.899923 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a944b83-f30e-4b95-9532-6617bc7d62db admin admin] PATH_INFO: `/v3/projects/71e259fdddc4480e9bfd48a768f7c2cb` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:45.901715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a944b83-f30e-4b95-9532-6617bc7d62db admin admin] RBAC: Authorizing `identity:delete_project(project_id=71e259fdddc4480e9bfd48a768f7c2cb)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:45.903758 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a944b83-f30e-4b95-9532-6617bc7d62db admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:45.915194 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ade5fc-545e-4978-aeab-69dce646f8c7 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:45.922356 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b08af-ad7f-4c01-ac04-dbb2b2fdab77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:45.923059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b08af-ad7f-4c01-ac04-dbb2b2fdab77 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:45.924145 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b08af-ad7f-4c01-ac04-dbb2b2fdab77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:45.924424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b08af-ad7f-4c01-ac04-dbb2b2fdab77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:45.925235 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b08af-ad7f-4c01-ac04-dbb2b2fdab77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:45.927195 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b08af-ad7f-4c01-ac04-dbb2b2fdab77 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:45.943298 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 976/1930] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:45.944819 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad417e9-95d9-4dec-b8aa-851b564d5136 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:45.945443 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad417e9-95d9-4dec-b8aa-851b564d5136 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:45.945443 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad417e9-95d9-4dec-b8aa-851b564d5136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:45.969512 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-9ad417e9-95d9-4dec-b8aa-851b564d5136 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:45.971269 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9ad417e9-95d9-4dec-b8aa-851b564d5136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:45.971604 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0a944b83-f30e-4b95-9532-6617bc7d62db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71e259fdddc4480e9bfd48a768f7c2cb', 'request_id': 'req-0a944b83-f30e-4b95-9532-6617bc7d62db'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:45.980022 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0a944b83-f30e-4b95-9532-6617bc7d62db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71e259fdddc4480e9bfd48a768f7c2cb', 'request_id': 'req-0a944b83-f30e-4b95-9532-6617bc7d62db'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:46.009626 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0a944b83-f30e-4b95-9532-6617bc7d62db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71e259fdddc4480e9bfd48a768f7c2cb', 'request_id': 'req-0a944b83-f30e-4b95-9532-6617bc7d62db'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:46.023798 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0a944b83-f30e-4b95-9532-6617bc7d62db admin admin] The token cache is being invalidate because project 71e259fdddc4480e9bfd48a768f7c2cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:31:46.024090 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0a944b83-f30e-4b95-9532-6617bc7d62db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:46.024749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0a944b83-f30e-4b95-9532-6617bc7d62db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:46.029703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 955/1931] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:31:45 2022] DELETE /identity/v3/projects/71e259fdddc4480e9bfd48a768f7c2cb => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:46.029703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a944b83-f30e-4b95-9532-6617bc7d62db admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:46.029703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0a944b83-f30e-4b95-9532-6617bc7d62db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:46.076161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 977/1932] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:46.078422 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f36c26-08af-46aa-a9b6-64c71cb7b0d4 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:46.078739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f36c26-08af-46aa-a9b6-64c71cb7b0d4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:46.079013 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f36c26-08af-46aa-a9b6-64c71cb7b0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:46.100516 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-39f36c26-08af-46aa-a9b6-64c71cb7b0d4 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:46.101724 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-39f36c26-08af-46aa-a9b6-64c71cb7b0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:46.116270 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 978/1933] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:46.116270 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253dcd4-0869-4b1a-99b0-a206615cffda None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:46.116270 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253dcd4-0869-4b1a-99b0-a206615cffda None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:46.116270 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253dcd4-0869-4b1a-99b0-a206615cffda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:46.122686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a944b83-f30e-4b95-9532-6617bc7d62db admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:46.128071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939dabb0-70ae-454d-b9f2-9be9f5287866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:46.128846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939dabb0-70ae-454d-b9f2-9be9f5287866 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:46.129135 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939dabb0-70ae-454d-b9f2-9be9f5287866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:46.129438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939dabb0-70ae-454d-b9f2-9be9f5287866 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:46.130212 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939dabb0-70ae-454d-b9f2-9be9f5287866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:46.132013 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939dabb0-70ae-454d-b9f2-9be9f5287866 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:46.132557 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-1253dcd4-0869-4b1a-99b0-a206615cffda None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:46.133817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1253dcd4-0869-4b1a-99b0-a206615cffda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:46.149143 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 979/1934] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:46.149330 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 956/1935] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:46.149424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb72fa2-20b9-4600-94c0-efe43a67184d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:46.149424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb72fa2-20b9-4600-94c0-efe43a67184d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:46.149424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb72fa2-20b9-4600-94c0-efe43a67184d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:46.150933 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939dabb0-70ae-454d-b9f2-9be9f5287866 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:46.161781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939dabb0-70ae-454d-b9f2-9be9f5287866 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:46.166846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd14f57-5607-49dd-874b-7e7ed535d98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:46.168057 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd14f57-5607-49dd-874b-7e7ed535d98d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:46.168468 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd14f57-5607-49dd-874b-7e7ed535d98d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:46.168750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd14f57-5607-49dd-874b-7e7ed535d98d admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:46.169233 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-9cb72fa2-20b9-4600-94c0-efe43a67184d None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:46.169399 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd14f57-5607-49dd-874b-7e7ed535d98d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:31:46.169704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd14f57-5607-49dd-874b-7e7ed535d98d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:46.170485 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9cb72fa2-20b9-4600-94c0-efe43a67184d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:46.171446 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd14f57-5607-49dd-874b-7e7ed535d98d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:46.178705 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 980/1936] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:46.182801 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb72fa2-20b9-4600-94c0-efe43a67184d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:46.185594 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 957/1937] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:31:46.186726 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd14f57-5607-49dd-874b-7e7ed535d98d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:46.193915 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb72fa2-20b9-4600-94c0-efe43a67184d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:46.198228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd14f57-5607-49dd-874b-7e7ed535d98d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:46.199899 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1363c0-3440-4b0a-8e62-adf6253bedbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:46.200668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1363c0-3440-4b0a-8e62-adf6253bedbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:46.200957 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1363c0-3440-4b0a-8e62-adf6253bedbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:46.201227 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1363c0-3440-4b0a-8e62-adf6253bedbd admin admin] PATH_INFO: `/v3/projects/668afd5bdf9f4e619ebbaadbb01292b8/users/ec23596b94234ef8a57d5a30836d3079/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:46.203966 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bee5de-f8f1-467a-9076-c1e851699d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:46.204566 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bee5de-f8f1-467a-9076-c1e851699d90 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:46.204847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bee5de-f8f1-467a-9076-c1e851699d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:46.205120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bee5de-f8f1-467a-9076-c1e851699d90 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:46.205774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bee5de-f8f1-467a-9076-c1e851699d90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:31:46.206085 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bee5de-f8f1-467a-9076-c1e851699d90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:46.207732 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bee5de-f8f1-467a-9076-c1e851699d90 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:46.217864 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:46.218030 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1363c0-3440-4b0a-8e62-adf6253bedbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=668afd5bdf9f4e619ebbaadbb01292b8, user_id=ec23596b94234ef8a57d5a30836d3079, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:46.221024 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1363c0-3440-4b0a-8e62-adf6253bedbd admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:46.221849 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 958/1938] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:31:46.223822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bee5de-f8f1-467a-9076-c1e851699d90 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:46.235759 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bee5de-f8f1-467a-9076-c1e851699d90 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:46.242361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 981/1939] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:46 2022] PUT /identity/v3/projects/668afd5bdf9f4e619ebbaadbb01292b8/users/ec23596b94234ef8a57d5a30836d3079/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:46.242361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1363c0-3440-4b0a-8e62-adf6253bedbd admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:46.247600 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:46.300615 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30b225-958f-44fc-b479-bcbdfbaed1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:46.301314 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30b225-958f-44fc-b479-bcbdfbaed1ea admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:46.301526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30b225-958f-44fc-b479-bcbdfbaed1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:46.301794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30b225-958f-44fc-b479-bcbdfbaed1ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:46.302452 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f30b225-958f-44fc-b479-bcbdfbaed1ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:46.304336 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f30b225-958f-44fc-b479-bcbdfbaed1ea admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:46.313974 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1363c0-3440-4b0a-8e62-adf6253bedbd admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:46.318616 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e934a-9965-442e-ab30-947fe123742f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:46.319217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e934a-9965-442e-ab30-947fe123742f admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:46.319498 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e934a-9965-442e-ab30-947fe123742f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:46.319775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e934a-9965-442e-ab30-947fe123742f admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:46.320402 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706e934a-9965-442e-ab30-947fe123742f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:46.322407 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706e934a-9965-442e-ab30-947fe123742f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:46.334214 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 959/1940] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:31:46 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:46.335575 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907535b9-08f1-49eb-8f13-a0044c4fddd3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:46.335869 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907535b9-08f1-49eb-8f13-a0044c4fddd3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:46.336150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907535b9-08f1-49eb-8f13-a0044c4fddd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:46.354007 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 982/1941] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:31:46 2022] POST /identity/v3/projects => generated 391 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:46.355172 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e934a-9965-442e-ab30-947fe123742f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:46.365338 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-907535b9-08f1-49eb-8f13-a0044c4fddd3 None None] MFA Rules not processed for user `ec23596b94234ef8a57d5a30836d3079`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:46.366649 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-907535b9-08f1-49eb-8f13-a0044c4fddd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:46.419985 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e934a-9965-442e-ab30-947fe123742f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:46.425573 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d6c67-c3d1-4407-aaae-8801fe840d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:46.426405 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d6c67-c3d1-4407-aaae-8801fe840d54 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:46.426689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d6c67-c3d1-4407-aaae-8801fe840d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:46.426970 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d6c67-c3d1-4407-aaae-8801fe840d54 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:46.427608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2d6c67-c3d1-4407-aaae-8801fe840d54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:46.429526 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2d6c67-c3d1-4407-aaae-8801fe840d54 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:46.434685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 960/1942] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:46.435592 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907535b9-08f1-49eb-8f13-a0044c4fddd3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:46.449580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907535b9-08f1-49eb-8f13-a0044c4fddd3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:46.452426 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 983/1943] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:31:46 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:46.453301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d6c67-c3d1-4407-aaae-8801fe840d54 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:46.456131 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db3a9d-7148-426f-882b-991a1382f9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:46.456742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db3a9d-7148-426f-882b-991a1382f9ba admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:46.457024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db3a9d-7148-426f-882b-991a1382f9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:46.457356 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db3a9d-7148-426f-882b-991a1382f9ba admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:46.457996 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db3a9d-7148-426f-882b-991a1382f9ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:46.459978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db3a9d-7148-426f-882b-991a1382f9ba admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:46.463264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d6c67-c3d1-4407-aaae-8801fe840d54 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:46.468622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f831e5-0a2d-4016-afb1-30043a249c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:46.469139 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f831e5-0a2d-4016-afb1-30043a249c03 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:46.469803 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f831e5-0a2d-4016-afb1-30043a249c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:46.469803 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f831e5-0a2d-4016-afb1-30043a249c03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:46.470327 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f831e5-0a2d-4016-afb1-30043a249c03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:46.472335 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f831e5-0a2d-4016-afb1-30043a249c03 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:46.485461 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 961/1944] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:31:46 2022] POST /identity/v3/users => generated 296 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:46.485461 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db3a9d-7148-426f-882b-991a1382f9ba admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:46.492433 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db3a9d-7148-426f-882b-991a1382f9ba admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:46.497665 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb5164-a784-4e8a-b185-b2f3522f777b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:46.498282 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb5164-a784-4e8a-b185-b2f3522f777b admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:46.498568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb5164-a784-4e8a-b185-b2f3522f777b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:46.498848 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb5164-a784-4e8a-b185-b2f3522f777b admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:46.499486 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb5164-a784-4e8a-b185-b2f3522f777b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:46.501082 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb5164-a784-4e8a-b185-b2f3522f777b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:46.504025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 984/1945] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:31:46 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:46.504835 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f831e5-0a2d-4016-afb1-30043a249c03 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:46.512481 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 962/1946] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:46.513276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb5164-a784-4e8a-b185-b2f3522f777b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:46.522860 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb5164-a784-4e8a-b185-b2f3522f777b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:46.568127 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:46.577094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f831e5-0a2d-4016-afb1-30043a249c03 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:46.581652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b8098-7148-4ced-a2f6-2d05bda6d889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:46.582217 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3afe1-7d43-4d6d-9de5-7e1900a6c935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:46.582339 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b8098-7148-4ced-a2f6-2d05bda6d889 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:46.582571 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b8098-7148-4ced-a2f6-2d05bda6d889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:46.582831 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3afe1-7d43-4d6d-9de5-7e1900a6c935 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:46.582927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b8098-7148-4ced-a2f6-2d05bda6d889 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:46.583117 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3afe1-7d43-4d6d-9de5-7e1900a6c935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:46.583396 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3afe1-7d43-4d6d-9de5-7e1900a6c935 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:46.583531 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b8098-7148-4ced-a2f6-2d05bda6d889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:46.584058 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3afe1-7d43-4d6d-9de5-7e1900a6c935 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:46.585071 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b8098-7148-4ced-a2f6-2d05bda6d889 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:46.585984 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3afe1-7d43-4d6d-9de5-7e1900a6c935 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:46.587318 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:46.597681 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 985/1947] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:46.598375 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b8098-7148-4ced-a2f6-2d05bda6d889 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:46.609672 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b8098-7148-4ced-a2f6-2d05bda6d889 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:46.614249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 963/1948] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:31:46 2022] POST /identity/v3/users => generated 301 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:46.614249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3afe1-7d43-4d6d-9de5-7e1900a6c935 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:46.619188 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4be5d-6456-431c-a866-36502cfa1f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:46.619641 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4be5d-6456-431c-a866-36502cfa1f97 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:46.619916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4be5d-6456-431c-a866-36502cfa1f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:46.620704 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4be5d-6456-431c-a866-36502cfa1f97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:46.621355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b4be5d-6456-431c-a866-36502cfa1f97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:46.622992 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b4be5d-6456-431c-a866-36502cfa1f97 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:46.625347 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3afe1-7d43-4d6d-9de5-7e1900a6c935 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:46.630208 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb6b39-9479-4f88-8989-9576933c947f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:46.630882 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb6b39-9479-4f88-8989-9576933c947f admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:46.631164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb6b39-9479-4f88-8989-9576933c947f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:46.631438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb6b39-9479-4f88-8989-9576933c947f admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:46.632107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb6b39-9479-4f88-8989-9576933c947f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:46.637327 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 986/1949] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:46.637455 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb6b39-9479-4f88-8989-9576933c947f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:46.637534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4be5d-6456-431c-a866-36502cfa1f97 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:46.647526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 964/1950] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:46.647695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4be5d-6456-431c-a866-36502cfa1f97 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:46.647813 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb6b39-9479-4f88-8989-9576933c947f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:46.651137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96785bdc-d1b0-4e87-9133-34e8d1271895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:46.651923 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96785bdc-d1b0-4e87-9133-34e8d1271895 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:46.652386 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96785bdc-d1b0-4e87-9133-34e8d1271895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:46.653405 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96785bdc-d1b0-4e87-9133-34e8d1271895 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:46.653569 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96785bdc-d1b0-4e87-9133-34e8d1271895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:46.656027 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96785bdc-d1b0-4e87-9133-34e8d1271895 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:46.659422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb6b39-9479-4f88-8989-9576933c947f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:46.664531 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6642e-193b-47de-93a1-450963ad2573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:46.665193 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6642e-193b-47de-93a1-450963ad2573 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:46.665483 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6642e-193b-47de-93a1-450963ad2573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:46.665765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6642e-193b-47de-93a1-450963ad2573 admin admin] PATH_INFO: `/v3/projects/f643701d94cf45b98130084f478e50b7/users/cb770cf7b9aa4968bd618a88f3a6af7b/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:46.672680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 987/1951] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:46.673455 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96785bdc-d1b0-4e87-9133-34e8d1271895 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:46.681562 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6642e-193b-47de-93a1-450963ad2573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f643701d94cf45b98130084f478e50b7, user_id=cb770cf7b9aa4968bd618a88f3a6af7b, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:46.684710 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96785bdc-d1b0-4e87-9133-34e8d1271895 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:46.684868 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6642e-193b-47de-93a1-450963ad2573 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:46.689410 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b8497-ad80-4129-b0d7-e0ea489192da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:46.690460 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b8497-ad80-4129-b0d7-e0ea489192da admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:46.690934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b8497-ad80-4129-b0d7-e0ea489192da admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:46.691286 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b8497-ad80-4129-b0d7-e0ea489192da admin admin] PATH_INFO: `/v3/projects/16b8dfcef75348d5888c9499be45ff64/users/a1aadd3dba014433bb6995e3cd3a029b/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:46.710668 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 965/1952] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:46 2022] PUT /identity/v3/projects/f643701d94cf45b98130084f478e50b7/users/cb770cf7b9aa4968bd618a88f3a6af7b/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:46.710668 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6642e-193b-47de-93a1-450963ad2573 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:46.725238 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b8497-ad80-4129-b0d7-e0ea489192da admin admin] RBAC: Authorizing `identity:create_grant(project_id=16b8dfcef75348d5888c9499be45ff64, user_id=a1aadd3dba014433bb6995e3cd3a029b, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:46.728816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b8497-ad80-4129-b0d7-e0ea489192da admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:46.750362 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 988/1953] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:46 2022] PUT /identity/v3/projects/16b8dfcef75348d5888c9499be45ff64/users/a1aadd3dba014433bb6995e3cd3a029b/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:46.752217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c1216-6630-4508-b4ac-bd082a073365 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:46.752633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c1216-6630-4508-b4ac-bd082a073365 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:46.753024 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c1216-6630-4508-b4ac-bd082a073365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:46.782703 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5b0c1216-6630-4508-b4ac-bd082a073365 None None] MFA Rules not processed for user `cb770cf7b9aa4968bd618a88f3a6af7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:46.784549 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5b0c1216-6630-4508-b4ac-bd082a073365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:46.796624 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6642e-193b-47de-93a1-450963ad2573 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:46.862616 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3cf7e2-d109-42d4-8edd-87eeec7a16be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:46.863417 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3cf7e2-d109-42d4-8edd-87eeec7a16be admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:46.863783 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3cf7e2-d109-42d4-8edd-87eeec7a16be admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:46.864070 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3cf7e2-d109-42d4-8edd-87eeec7a16be admin admin] PATH_INFO: `/v3/projects/0a426da6c08f4158b2d9103a4d5892c5/users/36b5f6c7a8b44d81b12c6e7d9c1973c3/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:46.867819 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 989/1954] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:46.869466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d4d2a-707c-44b3-9f27-329d2acf1a4c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:46.870004 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d4d2a-707c-44b3-9f27-329d2acf1a4c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:46.870970 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d4d2a-707c-44b3-9f27-329d2acf1a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:46.890000 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3cf7e2-d109-42d4-8edd-87eeec7a16be admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a426da6c08f4158b2d9103a4d5892c5, user_id=36b5f6c7a8b44d81b12c6e7d9c1973c3, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:46.894093 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3cf7e2-d109-42d4-8edd-87eeec7a16be admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:46.921545 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-917d4d2a-707c-44b3-9f27-329d2acf1a4c None None] MFA Rules not processed for user `a1aadd3dba014433bb6995e3cd3a029b`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:46.926839 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 966/1955] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:46 2022] PUT /identity/v3/projects/0a426da6c08f4158b2d9103a4d5892c5/users/36b5f6c7a8b44d81b12c6e7d9c1973c3/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:46.926984 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-917d4d2a-707c-44b3-9f27-329d2acf1a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:46.927072 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3cf7e2-d109-42d4-8edd-87eeec7a16be admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:46.977629 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:47.023917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3cf7e2-d109-42d4-8edd-87eeec7a16be admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:47.031872 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 990/1956] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:46 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:47.033093 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d4d2a-707c-44b3-9f27-329d2acf1a4c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:47.039995 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a7e58-805f-4c28-bef3-d1f101d372ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:47.040313 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a7e58-805f-4c28-bef3-d1f101d372ce admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:47.040605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a7e58-805f-4c28-bef3-d1f101d372ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:47.040872 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a7e58-805f-4c28-bef3-d1f101d372ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:47.041520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a7e58-805f-4c28-bef3-d1f101d372ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:47.045908 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a7e58-805f-4c28-bef3-d1f101d372ce admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:47.054365 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d4d2a-707c-44b3-9f27-329d2acf1a4c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:47.069844 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b191fc2-076f-4f28-ace5-bdbdebee7f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:47.070705 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b191fc2-076f-4f28-ace5-bdbdebee7f6c admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:47.071227 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b191fc2-076f-4f28-ace5-bdbdebee7f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:47.071597 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b191fc2-076f-4f28-ace5-bdbdebee7f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:47.072724 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b191fc2-076f-4f28-ace5-bdbdebee7f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:47.075154 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b191fc2-076f-4f28-ace5-bdbdebee7f6c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:47.095832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 991/1957] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:47.095832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b191fc2-076f-4f28-ace5-bdbdebee7f6c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:47.118307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 967/1958] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:31:46 2022] POST /identity/v3/projects => generated 385 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:47.118307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a7e58-805f-4c28-bef3-d1f101d372ce admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:47.121926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b191fc2-076f-4f28-ace5-bdbdebee7f6c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:47.161961 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:47.218408 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a7e58-805f-4c28-bef3-d1f101d372ce admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:47.223393 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00cae0-f6c4-488a-b420-1b4bb46a83bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:47.224521 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00cae0-f6c4-488a-b420-1b4bb46a83bf admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:47.224945 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb269473-a0c9-4d21-94b0-f0e1ddc1e5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:47.225289 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00cae0-f6c4-488a-b420-1b4bb46a83bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:47.225739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00cae0-f6c4-488a-b420-1b4bb46a83bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:47.226926 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb269473-a0c9-4d21-94b0-f0e1ddc1e5fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:47.226926 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb269473-a0c9-4d21-94b0-f0e1ddc1e5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:47.227117 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb269473-a0c9-4d21-94b0-f0e1ddc1e5fc admin admin] PATH_INFO: `/v3/projects/0a426da6c08f4158b2d9103a4d5892c5/users/36b5f6c7a8b44d81b12c6e7d9c1973c3/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:47.228239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00cae0-f6c4-488a-b420-1b4bb46a83bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:47.229027 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb269473-a0c9-4d21-94b0-f0e1ddc1e5fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a426da6c08f4158b2d9103a4d5892c5, user_id=36b5f6c7a8b44d81b12c6e7d9c1973c3, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:47.231449 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00cae0-f6c4-488a-b420-1b4bb46a83bf admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:47.232514 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb269473-a0c9-4d21-94b0-f0e1ddc1e5fc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:47.251570 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 968/1959] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:47 2022] PUT /identity/v3/projects/0a426da6c08f4158b2d9103a4d5892c5/users/36b5f6c7a8b44d81b12c6e7d9c1973c3/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:47.252973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb269473-a0c9-4d21-94b0-f0e1ddc1e5fc admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:47.270484 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 992/1960] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:31:47 2022] POST /identity/v3/projects => generated 393 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:47.271909 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6c053-eded-4fed-978a-899eb55644fc None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:47.272256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6c053-eded-4fed-978a-899eb55644fc None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:47.272597 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6c053-eded-4fed-978a-899eb55644fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:47.319675 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-9da6c053-eded-4fed-978a-899eb55644fc None None] MFA Rules not processed for user `36b5f6c7a8b44d81b12c6e7d9c1973c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:47.321170 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9da6c053-eded-4fed-978a-899eb55644fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:47.363897 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb269473-a0c9-4d21-94b0-f0e1ddc1e5fc admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:47.411742 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 993/1961] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:47 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:47.413371 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da6c053-eded-4fed-978a-899eb55644fc None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:47.478992 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:47.493028 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d714bae-25ba-4175-9caa-c0645e373631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:47.494075 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d714bae-25ba-4175-9caa-c0645e373631 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:47.494480 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d714bae-25ba-4175-9caa-c0645e373631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:47.494857 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d714bae-25ba-4175-9caa-c0645e373631 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:47.496216 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d714bae-25ba-4175-9caa-c0645e373631 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:47.500496 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d714bae-25ba-4175-9caa-c0645e373631 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:47.501155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da6c053-eded-4fed-978a-899eb55644fc None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:47.514831 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a925f8b-e92e-47ef-bfb5-383282975818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:47.515139 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a925f8b-e92e-47ef-bfb5-383282975818 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:47.515517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a925f8b-e92e-47ef-bfb5-383282975818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:47.515884 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a925f8b-e92e-47ef-bfb5-383282975818 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:47.516754 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a925f8b-e92e-47ef-bfb5-383282975818 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:47.520783 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a925f8b-e92e-47ef-bfb5-383282975818 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:47.531826 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 969/1962] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:31:47 2022] POST /identity/v3/users => generated 293 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:47.533042 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d714bae-25ba-4175-9caa-c0645e373631 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:47.536779 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1d714bae-25ba-4175-9caa-c0645e373631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:47.572317 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 994/1963] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:31:47 2022] POST /identity/v3/users => generated 303 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:47.573342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a925f8b-e92e-47ef-bfb5-383282975818 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:47.585921 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a925f8b-e92e-47ef-bfb5-383282975818 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:47.592572 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133bc158-8d7f-429d-948f-9b03e62aea87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:47.593197 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133bc158-8d7f-429d-948f-9b03e62aea87 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:47.593510 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133bc158-8d7f-429d-948f-9b03e62aea87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:47.594383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133bc158-8d7f-429d-948f-9b03e62aea87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:47.595333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133bc158-8d7f-429d-948f-9b03e62aea87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:47.598387 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133bc158-8d7f-429d-948f-9b03e62aea87 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:47.613513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 995/1964] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:47.614542 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133bc158-8d7f-429d-948f-9b03e62aea87 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:47.631487 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133bc158-8d7f-429d-948f-9b03e62aea87 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:47.647467 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d714bae-25ba-4175-9caa-c0645e373631 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:47.652242 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4025a-8703-4a7e-b179-18cfe4412fc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:47.653446 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a39ea-59c4-4853-b059-d266594b72b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:47.654077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a39ea-59c4-4853-b059-d266594b72b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:47.654227 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4025a-8703-4a7e-b179-18cfe4412fc1 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:47.654456 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a39ea-59c4-4853-b059-d266594b72b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:47.654738 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a39ea-59c4-4853-b059-d266594b72b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:47.654932 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4025a-8703-4a7e-b179-18cfe4412fc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:47.655365 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4025a-8703-4a7e-b179-18cfe4412fc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:47.655577 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a39ea-59c4-4853-b059-d266594b72b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:47.657735 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a39ea-59c4-4853-b059-d266594b72b2 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:47.662493 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:47.675466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 996/1965] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:47.677103 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a39ea-59c4-4853-b059-d266594b72b2 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:47.680526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf4025a-8703-4a7e-b179-18cfe4412fc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:47.681989 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf4025a-8703-4a7e-b179-18cfe4412fc1 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:47.690909 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a39ea-59c4-4853-b059-d266594b72b2 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:47.699605 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4165d12-c921-4f11-a0b9-2bca850fb1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:47.700451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4165d12-c921-4f11-a0b9-2bca850fb1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:47.700923 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4165d12-c921-4f11-a0b9-2bca850fb1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:47.701213 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4165d12-c921-4f11-a0b9-2bca850fb1f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:47.701946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4165d12-c921-4f11-a0b9-2bca850fb1f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:47.702802 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 970/1966] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:47 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:47.704889 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4025a-8703-4a7e-b179-18cfe4412fc1 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:47.717507 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4165d12-c921-4f11-a0b9-2bca850fb1f1 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:47.729669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4025a-8703-4a7e-b179-18cfe4412fc1 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:47.740250 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 997/1967] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:47.746283 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4165d12-c921-4f11-a0b9-2bca850fb1f1 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:47.752427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174c371-ce57-4cd3-9333-ae3d0bb4c213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:47.753115 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174c371-ce57-4cd3-9333-ae3d0bb4c213 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:47.753398 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174c371-ce57-4cd3-9333-ae3d0bb4c213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:47.753689 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174c371-ce57-4cd3-9333-ae3d0bb4c213 admin admin] PATH_INFO: `/v3/projects/8155c85e75b044f69144958b5335836d/users/6ca6f223b0384ce48821c16175037064/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:47.778253 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4165d12-c921-4f11-a0b9-2bca850fb1f1 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:47.795866 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b893a-598a-444c-bc33-dbd9561f4397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:47.796553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b893a-598a-444c-bc33-dbd9561f4397 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:47.796831 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b893a-598a-444c-bc33-dbd9561f4397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:47.797103 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b893a-598a-444c-bc33-dbd9561f4397 admin admin] PATH_INFO: `/v3/projects/966114545c704a9aa6815cbc2a05db31/users/99f95a35519243faaf4c3aeeba709578/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:47.809276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174c371-ce57-4cd3-9333-ae3d0bb4c213 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8155c85e75b044f69144958b5335836d, user_id=6ca6f223b0384ce48821c16175037064, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:47.821599 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174c371-ce57-4cd3-9333-ae3d0bb4c213 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:47.861659 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b893a-598a-444c-bc33-dbd9561f4397 admin admin] RBAC: Authorizing `identity:create_grant(project_id=966114545c704a9aa6815cbc2a05db31, user_id=99f95a35519243faaf4c3aeeba709578, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:47.862860 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 971/1968] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:47 2022] PUT /identity/v3/projects/8155c85e75b044f69144958b5335836d/users/6ca6f223b0384ce48821c16175037064/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:47.867140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174c371-ce57-4cd3-9333-ae3d0bb4c213 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:47.880428 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b893a-598a-444c-bc33-dbd9561f4397 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:47.936983 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 998/1969] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:47 2022] PUT /identity/v3/projects/966114545c704a9aa6815cbc2a05db31/users/99f95a35519243faaf4c3aeeba709578/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:47.938301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b893a-598a-444c-bc33-dbd9561f4397 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:48.060702 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174c371-ce57-4cd3-9333-ae3d0bb4c213 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:48.102320 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b893a-598a-444c-bc33-dbd9561f4397 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:48.115659 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c37318-f2a9-49f4-9a57-de83bfc200e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:48.116311 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c37318-f2a9-49f4-9a57-de83bfc200e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:48.116585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c37318-f2a9-49f4-9a57-de83bfc200e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:48.116855 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c37318-f2a9-49f4-9a57-de83bfc200e3 admin admin] PATH_INFO: `/v3/users/ec23596b94234ef8a57d5a30836d3079` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:48.118436 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c37318-f2a9-49f4-9a57-de83bfc200e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec23596b94234ef8a57d5a30836d3079)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:48.120880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c37318-f2a9-49f4-9a57-de83bfc200e3 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:48.161454 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7dfd57-044e-4f9e-aaa3-9f065b1c178d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:48.162251 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7dfd57-044e-4f9e-aaa3-9f065b1c178d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:48.162632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7dfd57-044e-4f9e-aaa3-9f065b1c178d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:48.163167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7dfd57-044e-4f9e-aaa3-9f065b1c178d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:48.163955 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7dfd57-044e-4f9e-aaa3-9f065b1c178d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:48.166259 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7dfd57-044e-4f9e-aaa3-9f065b1c178d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:48.180570 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 972/1970] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:48.182319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af7d45-5888-4622-8d2c-3af410982171 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:48.183293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af7d45-5888-4622-8d2c-3af410982171 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:48.183737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af7d45-5888-4622-8d2c-3af410982171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:48.217322 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-78c37318-f2a9-49f4-9a57-de83bfc200e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec23596b94234ef8a57d5a30836d3079'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:48.227542 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-78c37318-f2a9-49f4-9a57-de83bfc200e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec23596b94234ef8a57d5a30836d3079'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:48.228293 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-78c37318-f2a9-49f4-9a57-de83bfc200e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec23596b94234ef8a57d5a30836d3079'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:48.228922 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-78c37318-f2a9-49f4-9a57-de83bfc200e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec23596b94234ef8a57d5a30836d3079'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:48.229281 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:48.255972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-94af7d45-5888-4622-8d2c-3af410982171 None None] MFA Rules not processed for user `99f95a35519243faaf4c3aeeba709578`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:48.257302 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-94af7d45-5888-4622-8d2c-3af410982171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:48.263563 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-78c37318-f2a9-49f4-9a57-de83bfc200e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec23596b94234ef8a57d5a30836d3079'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:48.289227 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 999/1971] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:31:47 2022] DELETE /identity/v3/users/ec23596b94234ef8a57d5a30836d3079 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:48.290637 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c37318-f2a9-49f4-9a57-de83bfc200e3 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:48.294074 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-78c37318-f2a9-49f4-9a57-de83bfc200e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:48.347599 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 973/1972] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:48.349010 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ce404-ef93-4400-b5bc-e328d3dd7182 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:48.349392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ce404-ef93-4400-b5bc-e328d3dd7182 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:48.349781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ce404-ef93-4400-b5bc-e328d3dd7182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:48.375720 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a77ce404-ef93-4400-b5bc-e328d3dd7182 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:48.377672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a77ce404-ef93-4400-b5bc-e328d3dd7182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:48.399302 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:48.428372 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c37318-f2a9-49f4-9a57-de83bfc200e3 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:48.441973 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba09676-d66e-4ccf-aa52-833b859eaa1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:48.443270 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba09676-d66e-4ccf-aa52-833b859eaa1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:48.443701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba09676-d66e-4ccf-aa52-833b859eaa1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:48.444113 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba09676-d66e-4ccf-aa52-833b859eaa1c admin admin] PATH_INFO: `/v3/projects/8155c85e75b044f69144958b5335836d/users/6ca6f223b0384ce48821c16175037064/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:48.447477 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba09676-d66e-4ccf-aa52-833b859eaa1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8155c85e75b044f69144958b5335836d, user_id=6ca6f223b0384ce48821c16175037064, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:48.453702 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba09676-d66e-4ccf-aa52-833b859eaa1c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:48.459954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 974/1973] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:48.461461 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ce404-ef93-4400-b5bc-e328d3dd7182 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:48.463236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a77ce404-ef93-4400-b5bc-e328d3dd7182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:48.469021 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1000/1974] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:48 2022] PUT /identity/v3/projects/8155c85e75b044f69144958b5335836d/users/6ca6f223b0384ce48821c16175037064/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:48.477800 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ce404-ef93-4400-b5bc-e328d3dd7182 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:48.478860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba09676-d66e-4ccf-aa52-833b859eaa1c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:48.480784 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3ba09676-d66e-4ccf-aa52-833b859eaa1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:48.554509 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba09676-d66e-4ccf-aa52-833b859eaa1c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:48.558152 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0685628-706b-4381-afb5-ccf82136b7d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:48.558787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0685628-706b-4381-afb5-ccf82136b7d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:48.559066 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0685628-706b-4381-afb5-ccf82136b7d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:48.559334 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0685628-706b-4381-afb5-ccf82136b7d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:48.575384 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5607dd76-8156-4d4c-9650-a6c78baac346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:48.576038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607dd76-8156-4d4c-9650-a6c78baac346 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:48.576363 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607dd76-8156-4d4c-9650-a6c78baac346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:48.576643 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607dd76-8156-4d4c-9650-a6c78baac346 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:48.577282 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5607dd76-8156-4d4c-9650-a6c78baac346 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:48.579403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5607dd76-8156-4d4c-9650-a6c78baac346 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:48.611017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 975/1975] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:31:48 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:48.612929 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7df33-7b24-4847-b7c8-90a57f69f3dd None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:48.613301 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7df33-7b24-4847-b7c8-90a57f69f3dd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:48.613667 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7df33-7b24-4847-b7c8-90a57f69f3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:48.629924 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0685628-706b-4381-afb5-ccf82136b7d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:48.631551 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0685628-706b-4381-afb5-ccf82136b7d1 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:48.643134 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1001/1976] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:48.644014 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0685628-706b-4381-afb5-ccf82136b7d1 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:48.656358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0685628-706b-4381-afb5-ccf82136b7d1 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:48.660657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-bfe7df33-7b24-4847-b7c8-90a57f69f3dd None None] MFA Rules not processed for user `6ca6f223b0384ce48821c16175037064`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:48.666588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bfe7df33-7b24-4847-b7c8-90a57f69f3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:48.684404 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:48.748291 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b1bf8-dc3a-4f17-906e-277f9e14810d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:48.750588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b1bf8-dc3a-4f17-906e-277f9e14810d admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:48.750888 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b1bf8-dc3a-4f17-906e-277f9e14810d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:48.751167 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b1bf8-dc3a-4f17-906e-277f9e14810d admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:48.754627 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b1bf8-dc3a-4f17-906e-277f9e14810d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:48.756639 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b1bf8-dc3a-4f17-906e-277f9e14810d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:48.771792 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 976/1977] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:48 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:48.782993 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7df33-7b24-4847-b7c8-90a57f69f3dd None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:48.785936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bfe7df33-7b24-4847-b7c8-90a57f69f3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:48.793366 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1002/1978] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:31:48 2022] POST /identity/v3/users => generated 299 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:48.797256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b1bf8-dc3a-4f17-906e-277f9e14810d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:48.813955 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b1bf8-dc3a-4f17-906e-277f9e14810d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:48.820357 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a53c27-fa1e-44bc-a30f-f746e5baa8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:48.821334 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a53c27-fa1e-44bc-a30f-f746e5baa8a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:48.821627 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a53c27-fa1e-44bc-a30f-f746e5baa8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:48.821904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a53c27-fa1e-44bc-a30f-f746e5baa8a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:48.823028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a53c27-fa1e-44bc-a30f-f746e5baa8a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:48.826114 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a53c27-fa1e-44bc-a30f-f746e5baa8a9 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:48.843607 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1003/1979] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:48.847132 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a53c27-fa1e-44bc-a30f-f746e5baa8a9 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:48.856594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:48.863361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a53c27-fa1e-44bc-a30f-f746e5baa8a9 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:48.870863 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33386de7-3f39-4495-a37b-d081b25b370b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:48.872436 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33386de7-3f39-4495-a37b-d081b25b370b admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:48.872774 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33386de7-3f39-4495-a37b-d081b25b370b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:48.873065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33386de7-3f39-4495-a37b-d081b25b370b admin admin] PATH_INFO: `/v3/projects/d686e2d585f849dc9832831136dfa705/users/c152e984d9424a2fb8764c35177410ac/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:48.878911 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7df33-7b24-4847-b7c8-90a57f69f3dd None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:48.884055 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e27b6-c63a-42ea-9a08-73e2a6118536 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:48.885144 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e27b6-c63a-42ea-9a08-73e2a6118536 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:48.885493 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e27b6-c63a-42ea-9a08-73e2a6118536 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:48.886369 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e27b6-c63a-42ea-9a08-73e2a6118536 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:48.897997 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e27b6-c63a-42ea-9a08-73e2a6118536 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:48.899614 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e27b6-c63a-42ea-9a08-73e2a6118536 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:48.916266 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 977/1980] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:48 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:31:48.938615 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33386de7-3f39-4495-a37b-d081b25b370b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d686e2d585f849dc9832831136dfa705, user_id=c152e984d9424a2fb8764c35177410ac, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:48.941646 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33386de7-3f39-4495-a37b-d081b25b370b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:48.986752 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e27b6-c63a-42ea-9a08-73e2a6118536 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:48.987222 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d67e27b6-c63a-42ea-9a08-73e2a6118536 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:48.988559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1004/1981] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:48 2022] PUT /identity/v3/projects/d686e2d585f849dc9832831136dfa705/users/c152e984d9424a2fb8764c35177410ac/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:31:48.992334 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33386de7-3f39-4495-a37b-d081b25b370b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:49.005362 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e27b6-c63a-42ea-9a08-73e2a6118536 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:49.073114 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33386de7-3f39-4495-a37b-d081b25b370b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:49.078415 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690afecb-ee74-4104-9d50-49b068915cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:49.080102 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690afecb-ee74-4104-9d50-49b068915cef admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:49.080552 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690afecb-ee74-4104-9d50-49b068915cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:49.080973 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690afecb-ee74-4104-9d50-49b068915cef admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:49.082286 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690afecb-ee74-4104-9d50-49b068915cef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:49.084824 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690afecb-ee74-4104-9d50-49b068915cef admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:49.093605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe449adc-db5a-45c1-961a-a6b182664592 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:49.094797 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe449adc-db5a-45c1-961a-a6b182664592 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:49.095311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe449adc-db5a-45c1-961a-a6b182664592 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:49.095894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe449adc-db5a-45c1-961a-a6b182664592 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:49.098234 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1005/1982] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:49.099218 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fe449adc-db5a-45c1-961a-a6b182664592 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:49.101959 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690afecb-ee74-4104-9d50-49b068915cef admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:49.117388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690afecb-ee74-4104-9d50-49b068915cef admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:49.125389 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845e4f1-6be3-42f7-bccd-41370d756e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:49.126173 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845e4f1-6be3-42f7-bccd-41370d756e95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:49.126590 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845e4f1-6be3-42f7-bccd-41370d756e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:49.127081 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845e4f1-6be3-42f7-bccd-41370d756e95 admin admin] PATH_INFO: `/v3/projects/d686e2d585f849dc9832831136dfa705/users/c152e984d9424a2fb8764c35177410ac/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:49.132074 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845e4f1-6be3-42f7-bccd-41370d756e95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d686e2d585f849dc9832831136dfa705, user_id=c152e984d9424a2fb8764c35177410ac, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:49.136404 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845e4f1-6be3-42f7-bccd-41370d756e95 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:49.157331 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1006/1983] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:49 2022] PUT /identity/v3/projects/d686e2d585f849dc9832831136dfa705/users/c152e984d9424a2fb8764c35177410ac/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:31:49.162422 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845e4f1-6be3-42f7-bccd-41370d756e95 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:49.162422 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9845e4f1-6be3-42f7-bccd-41370d756e95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:49.180059 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe449adc-db5a-45c1-961a-a6b182664592 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:49.181624 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe449adc-db5a-45c1-961a-a6b182664592 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:49.193928 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 978/1984] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:48 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:31:49.196705 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee39b91-e30c-417c-b0bf-9724cf81b325 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:49.197196 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee39b91-e30c-417c-b0bf-9724cf81b325 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:49.197805 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee39b91-e30c-417c-b0bf-9724cf81b325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:49.215417 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:49.234226 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0ee39b91-e30c-417c-b0bf-9724cf81b325 None None] MFA Rules not processed for user `c152e984d9424a2fb8764c35177410ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:49.235655 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0ee39b91-e30c-417c-b0bf-9724cf81b325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:49.256758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845e4f1-6be3-42f7-bccd-41370d756e95 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:49.261996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de3ad2-4fd4-4a94-a26f-51be0a5f35a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:49.262661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de3ad2-4fd4-4a94-a26f-51be0a5f35a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:49.262943 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de3ad2-4fd4-4a94-a26f-51be0a5f35a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:49.263214 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de3ad2-4fd4-4a94-a26f-51be0a5f35a4 admin admin] PATH_INFO: `/v3/projects/668afd5bdf9f4e619ebbaadbb01292b8` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:49.264371 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de3ad2-4fd4-4a94-a26f-51be0a5f35a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=668afd5bdf9f4e619ebbaadbb01292b8)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:49.266300 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de3ad2-4fd4-4a94-a26f-51be0a5f35a4 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:49.321263 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 979/1985] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:49 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:49.324326 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a5de3ad2-4fd4-4a94-a26f-51be0a5f35a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '668afd5bdf9f4e619ebbaadbb01292b8', 'request_id': 'req-a5de3ad2-4fd4-4a94-a26f-51be0a5f35a4'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:49.331132 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a5de3ad2-4fd4-4a94-a26f-51be0a5f35a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '668afd5bdf9f4e619ebbaadbb01292b8', 'request_id': 'req-a5de3ad2-4fd4-4a94-a26f-51be0a5f35a4'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:49.333695 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee39b91-e30c-417c-b0bf-9724cf81b325 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:49.338420 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a5de3ad2-4fd4-4a94-a26f-51be0a5f35a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '668afd5bdf9f4e619ebbaadbb01292b8', 'request_id': 'req-a5de3ad2-4fd4-4a94-a26f-51be0a5f35a4'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:49.342286 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:49.355379 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a5de3ad2-4fd4-4a94-a26f-51be0a5f35a4 admin admin] The token cache is being invalidate because project 668afd5bdf9f4e619ebbaadbb01292b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:31:49.355968 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a5de3ad2-4fd4-4a94-a26f-51be0a5f35a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:49.357175 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a5de3ad2-4fd4-4a94-a26f-51be0a5f35a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:49.359722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1007/1986] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:31:49 2022] DELETE /identity/v3/projects/668afd5bdf9f4e619ebbaadbb01292b8 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:49.365542 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de3ad2-4fd4-4a94-a26f-51be0a5f35a4 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:49.367257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a5de3ad2-4fd4-4a94-a26f-51be0a5f35a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:49.400236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee39b91-e30c-417c-b0bf-9724cf81b325 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:49.465284 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de3ad2-4fd4-4a94-a26f-51be0a5f35a4 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:49.467676 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a62ef2-9597-4603-8537-e16d123a651a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:49.468529 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a62ef2-9597-4603-8537-e16d123a651a service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:49.468983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a62ef2-9597-4603-8537-e16d123a651a service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:49.469302 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a62ef2-9597-4603-8537-e16d123a651a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:49.470081 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a5eba-69ba-4bda-9ee9-430373da1e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:49.470742 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a5eba-69ba-4bda-9ee9-430373da1e32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:49.471252 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a5eba-69ba-4bda-9ee9-430373da1e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:49.471464 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f1a62ef2-9597-4603-8537-e16d123a651a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:49.471580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a5eba-69ba-4bda-9ee9-430373da1e32 admin admin] PATH_INFO: `/v3/users/36b5f6c7a8b44d81b12c6e7d9c1973c3` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:49.472661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a5eba-69ba-4bda-9ee9-430373da1e32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36b5f6c7a8b44d81b12c6e7d9c1973c3)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:49.474602 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a5eba-69ba-4bda-9ee9-430373da1e32 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:49.557941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a62ef2-9597-4603-8537-e16d123a651a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:49.560144 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a62ef2-9597-4603-8537-e16d123a651a service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:49.578497 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1c5a5eba-69ba-4bda-9ee9-430373da1e32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36b5f6c7a8b44d81b12c6e7d9c1973c3'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:49.586383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1c5a5eba-69ba-4bda-9ee9-430373da1e32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36b5f6c7a8b44d81b12c6e7d9c1973c3'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:49.587147 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1c5a5eba-69ba-4bda-9ee9-430373da1e32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36b5f6c7a8b44d81b12c6e7d9c1973c3'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:49.587837 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 980/1987] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:31:49.587964 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1c5a5eba-69ba-4bda-9ee9-430373da1e32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36b5f6c7a8b44d81b12c6e7d9c1973c3'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:49.604139 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1c5a5eba-69ba-4bda-9ee9-430373da1e32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36b5f6c7a8b44d81b12c6e7d9c1973c3'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:49.625840 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1008/1988] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:31:49 2022] DELETE /identity/v3/users/36b5f6c7a8b44d81b12c6e7d9c1973c3 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:50.087629 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a62ef2-9597-4603-8537-e16d123a651a service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:50.094029 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:50.100969 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a62ef2-9597-4603-8537-e16d123a651a service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:50.169093 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1706a-f53f-4bb1-ba1c-35e32a7ee750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:50.169877 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1706a-f53f-4bb1-ba1c-35e32a7ee750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:50.170184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1706a-f53f-4bb1-ba1c-35e32a7ee750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:50.170485 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1706a-f53f-4bb1-ba1c-35e32a7ee750 admin admin] PATH_INFO: `/v3/projects/0a426da6c08f4158b2d9103a4d5892c5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:50.171620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c1706a-f53f-4bb1-ba1c-35e32a7ee750 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a426da6c08f4158b2d9103a4d5892c5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:50.173510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c1706a-f53f-4bb1-ba1c-35e32a7ee750 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:50.235201 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e5c1706a-f53f-4bb1-ba1c-35e32a7ee750 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a426da6c08f4158b2d9103a4d5892c5', 'request_id': 'req-e5c1706a-f53f-4bb1-ba1c-35e32a7ee750'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:50.241635 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e5c1706a-f53f-4bb1-ba1c-35e32a7ee750 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a426da6c08f4158b2d9103a4d5892c5', 'request_id': 'req-e5c1706a-f53f-4bb1-ba1c-35e32a7ee750'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:50.253154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e5c1706a-f53f-4bb1-ba1c-35e32a7ee750 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a426da6c08f4158b2d9103a4d5892c5', 'request_id': 'req-e5c1706a-f53f-4bb1-ba1c-35e32a7ee750'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:50.265484 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e5c1706a-f53f-4bb1-ba1c-35e32a7ee750 admin admin] The token cache is being invalidate because project 0a426da6c08f4158b2d9103a4d5892c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:31:50.265778 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e5c1706a-f53f-4bb1-ba1c-35e32a7ee750 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:50.266460 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e5c1706a-f53f-4bb1-ba1c-35e32a7ee750 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:50.268044 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 981/1989] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:31:50 2022] DELETE /identity/v3/projects/0a426da6c08f4158b2d9103a4d5892c5 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:50.273075 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d6c8d-107f-44bc-8298-9d8c3304f6d0 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:50.273385 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d6c8d-107f-44bc-8298-9d8c3304f6d0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:50.273664 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d6c8d-107f-44bc-8298-9d8c3304f6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:50.294728 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ed4d6c8d-107f-44bc-8298-9d8c3304f6d0 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:50.295836 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ed4d6c8d-107f-44bc-8298-9d8c3304f6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:50.358258 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1009/1990] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:50.366026 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e711cc7-a28d-45a4-8c68-4858aade67e1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:50.366420 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e711cc7-a28d-45a4-8c68-4858aade67e1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:50.366954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e711cc7-a28d-45a4-8c68-4858aade67e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:50.387225 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2e711cc7-a28d-45a4-8c68-4858aade67e1 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:50.388434 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2e711cc7-a28d-45a4-8c68-4858aade67e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:50.396393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 982/1991] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:50.400979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d6c8d-107f-44bc-8298-9d8c3304f6d0 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:50.414293 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d6c8d-107f-44bc-8298-9d8c3304f6d0 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:50.416747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654a9f8-c3c8-42db-b270-a353d7d56e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:50.417893 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654a9f8-c3c8-42db-b270-a353d7d56e46 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:50.418217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654a9f8-c3c8-42db-b270-a353d7d56e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:50.418481 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654a9f8-c3c8-42db-b270-a353d7d56e46 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:50.419140 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f654a9f8-c3c8-42db-b270-a353d7d56e46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:31:50.419445 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f654a9f8-c3c8-42db-b270-a353d7d56e46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:50.421042 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f654a9f8-c3c8-42db-b270-a353d7d56e46 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:50.429915 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:50.433345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1010/1992] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:31:50.438077 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e711cc7-a28d-45a4-8c68-4858aade67e1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:50.447489 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e711cc7-a28d-45a4-8c68-4858aade67e1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:50.452245 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cac7f3-08a5-4257-abd6-382e295ac444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:50.452848 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cac7f3-08a5-4257-abd6-382e295ac444 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:50.453158 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cac7f3-08a5-4257-abd6-382e295ac444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:50.453454 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cac7f3-08a5-4257-abd6-382e295ac444 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:50.454120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cac7f3-08a5-4257-abd6-382e295ac444 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:50.456052 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cac7f3-08a5-4257-abd6-382e295ac444 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:50.484282 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 983/1993] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:31:50 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:31:50.489184 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654a9f8-c3c8-42db-b270-a353d7d56e46 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:50.499096 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654a9f8-c3c8-42db-b270-a353d7d56e46 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:50.555464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90327ee-c929-4afb-964a-e172c7a017db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:50.555984 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90327ee-c929-4afb-964a-e172c7a017db admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:50.556261 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90327ee-c929-4afb-964a-e172c7a017db admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:50.556533 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90327ee-c929-4afb-964a-e172c7a017db admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:50.557151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90327ee-c929-4afb-964a-e172c7a017db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:50.559146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90327ee-c929-4afb-964a-e172c7a017db admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:50.582229 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1011/1994] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:31:50 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:50.586284 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cac7f3-08a5-4257-abd6-382e295ac444 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:50.597004 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cac7f3-08a5-4257-abd6-382e295ac444 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:50.601509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b45626-48ad-4627-a86a-a1bb9bee0365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:50.602103 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b45626-48ad-4627-a86a-a1bb9bee0365 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:50.602416 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b45626-48ad-4627-a86a-a1bb9bee0365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:50.602695 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b45626-48ad-4627-a86a-a1bb9bee0365 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:50.603341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b45626-48ad-4627-a86a-a1bb9bee0365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:50.605573 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b45626-48ad-4627-a86a-a1bb9bee0365 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:50.618059 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 984/1995] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:50.619186 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90327ee-c929-4afb-964a-e172c7a017db admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:50.630029 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90327ee-c929-4afb-964a-e172c7a017db admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:50.634656 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c9e43-94a9-4357-9b2b-1cdf00b961b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:50.635414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c9e43-94a9-4357-9b2b-1cdf00b961b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:50.635807 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c9e43-94a9-4357-9b2b-1cdf00b961b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:50.636140 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c9e43-94a9-4357-9b2b-1cdf00b961b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:50.636848 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c9e43-94a9-4357-9b2b-1cdf00b961b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:50.638516 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c9e43-94a9-4357-9b2b-1cdf00b961b3 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:50.648972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1012/1996] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:50.653768 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b45626-48ad-4627-a86a-a1bb9bee0365 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:50.664195 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b45626-48ad-4627-a86a-a1bb9bee0365 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:50.669541 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a2cb06-9f76-466a-86e2-eecb112cbceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:50.670217 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2cb06-9f76-466a-86e2-eecb112cbceb admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:50.670510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2cb06-9f76-466a-86e2-eecb112cbceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:50.670787 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2cb06-9f76-466a-86e2-eecb112cbceb admin admin] PATH_INFO: `/v3/projects/fd8e0ed266414e2c8c2f177746b579da/users/9f93a51f82a94e269bca847a4f629bf6/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:50.685626 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a2cb06-9f76-466a-86e2-eecb112cbceb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd8e0ed266414e2c8c2f177746b579da, user_id=9f93a51f82a94e269bca847a4f629bf6, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:50.688422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a2cb06-9f76-466a-86e2-eecb112cbceb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:50.702834 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 985/1997] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:50 2022] PUT /identity/v3/projects/fd8e0ed266414e2c8c2f177746b579da/users/9f93a51f82a94e269bca847a4f629bf6/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:50.714614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef8c3e-3e03-41f3-aba1-3cf6517a28f3 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:50.714951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef8c3e-3e03-41f3-aba1-3cf6517a28f3 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:50.715175 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef8c3e-3e03-41f3-aba1-3cf6517a28f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:50.743507 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-29ef8c3e-3e03-41f3-aba1-3cf6517a28f3 None None] MFA Rules not processed for user `9f93a51f82a94e269bca847a4f629bf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:50.744713 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-29ef8c3e-3e03-41f3-aba1-3cf6517a28f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:50.817377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1013/1998] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:50.837530 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a2cb06-9f76-466a-86e2-eecb112cbceb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:50.839294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-29a2cb06-9f76-466a-86e2-eecb112cbceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:50.881360 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:50.911408 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a2cb06-9f76-466a-86e2-eecb112cbceb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:50.915517 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d011164-8500-425d-870a-4973a717a41c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:50.916127 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d011164-8500-425d-870a-4973a717a41c service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:50.916408 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d011164-8500-425d-870a-4973a717a41c service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:50.916688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d011164-8500-425d-870a-4973a717a41c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:50.926661 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d011164-8500-425d-870a-4973a717a41c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:50.928161 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d011164-8500-425d-870a-4973a717a41c service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:50.939008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 986/1999] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:51.447717 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef8c3e-3e03-41f3-aba1-3cf6517a28f3 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:51.458323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef8c3e-3e03-41f3-aba1-3cf6517a28f3 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:51.463120 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf5b43-8fda-435e-a3bf-2e5ab3f54777 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:51.464048 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf5b43-8fda-435e-a3bf-2e5ab3f54777 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:51.464256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf5b43-8fda-435e-a3bf-2e5ab3f54777 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:51.464617 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf5b43-8fda-435e-a3bf-2e5ab3f54777 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:51.466645 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5dcf5b43-8fda-435e-a3bf-2e5ab3f54777 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:51.550086 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcf5b43-8fda-435e-a3bf-2e5ab3f54777 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:51.551992 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcf5b43-8fda-435e-a3bf-2e5ab3f54777 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:51.563969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1014/2000] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:52.168256 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d011164-8500-425d-870a-4973a717a41c service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:52.179571 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d011164-8500-425d-870a-4973a717a41c service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:52.183971 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bd09e-3fe6-4be8-84f3-327041aa0669 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:52.184639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bd09e-3fe6-4be8-84f3-327041aa0669 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:52.184993 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bd09e-3fe6-4be8-84f3-327041aa0669 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:52.185325 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bd09e-3fe6-4be8-84f3-327041aa0669 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:52.187349 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3a3bd09e-3fe6-4be8-84f3-327041aa0669 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:52.257577 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3bd09e-3fe6-4be8-84f3-327041aa0669 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:52.259049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3bd09e-3fe6-4be8-84f3-327041aa0669 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:52.270538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 987/2001] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:52 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:53.982740 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf5b43-8fda-435e-a3bf-2e5ab3f54777 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:53.984179 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5dcf5b43-8fda-435e-a3bf-2e5ab3f54777 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:53.994879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:54.055927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf5b43-8fda-435e-a3bf-2e5ab3f54777 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:54.060551 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018c1dd-a33e-4fd4-949f-dcfd3c1c287c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:54.061185 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018c1dd-a33e-4fd4-949f-dcfd3c1c287c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:54.061464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018c1dd-a33e-4fd4-949f-dcfd3c1c287c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:54.061737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018c1dd-a33e-4fd4-949f-dcfd3c1c287c admin admin] PATH_INFO: `/v3/users/a1aadd3dba014433bb6995e3cd3a029b` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:54.062890 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9018c1dd-a33e-4fd4-949f-dcfd3c1c287c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1aadd3dba014433bb6995e3cd3a029b)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:54.064818 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9018c1dd-a33e-4fd4-949f-dcfd3c1c287c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:54.112402 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-9018c1dd-a33e-4fd4-949f-dcfd3c1c287c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1aadd3dba014433bb6995e3cd3a029b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:54.119787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-9018c1dd-a33e-4fd4-949f-dcfd3c1c287c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1aadd3dba014433bb6995e3cd3a029b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:54.120471 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-9018c1dd-a33e-4fd4-949f-dcfd3c1c287c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1aadd3dba014433bb6995e3cd3a029b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:54.121089 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-9018c1dd-a33e-4fd4-949f-dcfd3c1c287c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1aadd3dba014433bb6995e3cd3a029b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:54.133761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-9018c1dd-a33e-4fd4-949f-dcfd3c1c287c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1aadd3dba014433bb6995e3cd3a029b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:54.146483 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1015/2002] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:31:53 2022] DELETE /identity/v3/users/a1aadd3dba014433bb6995e3cd3a029b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:54.151528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4a907-de15-4f11-bef7-382b4d5c2a55 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:54.152013 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4a907-de15-4f11-bef7-382b4d5c2a55 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:54.152111 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4a907-de15-4f11-bef7-382b4d5c2a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:54.171640 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ede4a907-de15-4f11-bef7-382b4d5c2a55 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:54.172846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ede4a907-de15-4f11-bef7-382b4d5c2a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:54.233636 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 988/2003] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:54.243815 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018c1dd-a33e-4fd4-949f-dcfd3c1c287c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:54.245361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9018c1dd-a33e-4fd4-949f-dcfd3c1c287c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:54.315315 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018c1dd-a33e-4fd4-949f-dcfd3c1c287c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:54.319035 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38bf6ee-4974-48ca-aba3-b180f3367dd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:54.319643 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bf6ee-4974-48ca-aba3-b180f3367dd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:54.319924 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bf6ee-4974-48ca-aba3-b180f3367dd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:54.320198 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bf6ee-4974-48ca-aba3-b180f3367dd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:54.330678 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38bf6ee-4974-48ca-aba3-b180f3367dd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:54.332112 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38bf6ee-4974-48ca-aba3-b180f3367dd3 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:54.342967 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1016/2004] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:54.634238 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4a907-de15-4f11-bef7-382b4d5c2a55 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:54.636214 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ede4a907-de15-4f11-bef7-382b4d5c2a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:54.648722 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4a907-de15-4f11-bef7-382b4d5c2a55 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:54.654019 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba59d4-0f29-4654-a47e-a7687a336c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:54.654692 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba59d4-0f29-4654-a47e-a7687a336c9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:54.654975 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba59d4-0f29-4654-a47e-a7687a336c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:54.655250 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba59d4-0f29-4654-a47e-a7687a336c9d admin admin] PATH_INFO: `/v3/projects/16b8dfcef75348d5888c9499be45ff64` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:54.656396 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba59d4-0f29-4654-a47e-a7687a336c9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=16b8dfcef75348d5888c9499be45ff64)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:54.658489 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba59d4-0f29-4654-a47e-a7687a336c9d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:54.720126 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-efba59d4-0f29-4654-a47e-a7687a336c9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16b8dfcef75348d5888c9499be45ff64', 'request_id': 'req-efba59d4-0f29-4654-a47e-a7687a336c9d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:54.727287 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-efba59d4-0f29-4654-a47e-a7687a336c9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16b8dfcef75348d5888c9499be45ff64', 'request_id': 'req-efba59d4-0f29-4654-a47e-a7687a336c9d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:54.737564 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-efba59d4-0f29-4654-a47e-a7687a336c9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16b8dfcef75348d5888c9499be45ff64', 'request_id': 'req-efba59d4-0f29-4654-a47e-a7687a336c9d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:54.739742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:54.778495 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-efba59d4-0f29-4654-a47e-a7687a336c9d admin admin] The token cache is being invalidate because project 16b8dfcef75348d5888c9499be45ff64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:31:54.778802 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-efba59d4-0f29-4654-a47e-a7687a336c9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:54.779490 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-efba59d4-0f29-4654-a47e-a7687a336c9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:54.781388 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 989/2005] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:31:54 2022] DELETE /identity/v3/projects/16b8dfcef75348d5888c9499be45ff64 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:54.785237 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38bf6ee-4974-48ca-aba3-b180f3367dd3 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:54.798784 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38bf6ee-4974-48ca-aba3-b180f3367dd3 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:54.826749 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:54.861264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671af286-d019-4abe-8172-897e04a8561e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:54.861895 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671af286-d019-4abe-8172-897e04a8561e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:54.862178 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671af286-d019-4abe-8172-897e04a8561e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:54.862494 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671af286-d019-4abe-8172-897e04a8561e admin admin] PATH_INFO: `/v3/users/6ca6f223b0384ce48821c16175037064` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:54.863616 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671af286-d019-4abe-8172-897e04a8561e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ca6f223b0384ce48821c16175037064)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:54.865550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671af286-d019-4abe-8172-897e04a8561e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:54.914008 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-671af286-d019-4abe-8172-897e04a8561e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ca6f223b0384ce48821c16175037064'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:54.921886 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-671af286-d019-4abe-8172-897e04a8561e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ca6f223b0384ce48821c16175037064'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:54.922563 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-671af286-d019-4abe-8172-897e04a8561e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ca6f223b0384ce48821c16175037064'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:54.923278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-671af286-d019-4abe-8172-897e04a8561e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ca6f223b0384ce48821c16175037064'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:54.936768 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-671af286-d019-4abe-8172-897e04a8561e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ca6f223b0384ce48821c16175037064'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:54.949967 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1017/2006] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:31:54 2022] DELETE /identity/v3/users/6ca6f223b0384ce48821c16175037064 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:55.221923 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba59d4-0f29-4654-a47e-a7687a336c9d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:55.223871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-efba59d4-0f29-4654-a47e-a7687a336c9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:55.237272 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba59d4-0f29-4654-a47e-a7687a336c9d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:55.295434 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2ea00-9e27-4be1-875a-9aec28159b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:55.295987 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2ea00-9e27-4be1-875a-9aec28159b8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:55.296266 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2ea00-9e27-4be1-875a-9aec28159b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:55.296540 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2ea00-9e27-4be1-875a-9aec28159b8c admin admin] PATH_INFO: `/v3/projects/8155c85e75b044f69144958b5335836d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:55.297642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f2ea00-9e27-4be1-875a-9aec28159b8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8155c85e75b044f69144958b5335836d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:55.299581 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f2ea00-9e27-4be1-875a-9aec28159b8c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:55.345770 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3f2ea00-9e27-4be1-875a-9aec28159b8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8155c85e75b044f69144958b5335836d', 'request_id': 'req-a3f2ea00-9e27-4be1-875a-9aec28159b8c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:55.351790 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3f2ea00-9e27-4be1-875a-9aec28159b8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8155c85e75b044f69144958b5335836d', 'request_id': 'req-a3f2ea00-9e27-4be1-875a-9aec28159b8c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:55.359177 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3f2ea00-9e27-4be1-875a-9aec28159b8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8155c85e75b044f69144958b5335836d', 'request_id': 'req-a3f2ea00-9e27-4be1-875a-9aec28159b8c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:55.369355 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3f2ea00-9e27-4be1-875a-9aec28159b8c admin admin] The token cache is being invalidate because project 8155c85e75b044f69144958b5335836d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:31:55.369645 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3f2ea00-9e27-4be1-875a-9aec28159b8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:55.370336 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3f2ea00-9e27-4be1-875a-9aec28159b8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:31:55.371802 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 990/2007] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:31:55 2022] DELETE /identity/v3/projects/8155c85e75b044f69144958b5335836d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:55.377799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59457e1c-eee7-4be5-b0bf-c899cd0c6945 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:55.378375 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59457e1c-eee7-4be5-b0bf-c899cd0c6945 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:55.378492 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59457e1c-eee7-4be5-b0bf-c899cd0c6945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:55.398656 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-59457e1c-eee7-4be5-b0bf-c899cd0c6945 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:55.399850 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-59457e1c-eee7-4be5-b0bf-c899cd0c6945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:55.459822 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1018/2008] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:55.467648 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f70d1-14b8-4778-9334-3105e5ef95ea None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:55.467935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f70d1-14b8-4778-9334-3105e5ef95ea None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:55.468208 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f70d1-14b8-4778-9334-3105e5ef95ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:55.488169 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9f4f70d1-14b8-4778-9334-3105e5ef95ea None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:55.489496 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9f4f70d1-14b8-4778-9334-3105e5ef95ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:55.498227 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 991/2009] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:55.502489 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59457e1c-eee7-4be5-b0bf-c899cd0c6945 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:55.512614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59457e1c-eee7-4be5-b0bf-c899cd0c6945 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:55.517186 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a59b2-068f-4707-8fd0-a7d530476634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:55.517778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a59b2-068f-4707-8fd0-a7d530476634 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:55.518059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a59b2-068f-4707-8fd0-a7d530476634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:55.518366 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a59b2-068f-4707-8fd0-a7d530476634 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:55.519337 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a59b2-068f-4707-8fd0-a7d530476634 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:31:55.519647 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a59b2-068f-4707-8fd0-a7d530476634 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:55.521258 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a59b2-068f-4707-8fd0-a7d530476634 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:55.533899 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1019/2010] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:31:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:31:55.538280 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f70d1-14b8-4778-9334-3105e5ef95ea None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:55.548303 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f70d1-14b8-4778-9334-3105e5ef95ea None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:55.553684 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdcfe3-9ad5-47a5-8868-aa8c90b413a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:55.554321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdcfe3-9ad5-47a5-8868-aa8c90b413a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:55.554604 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdcfe3-9ad5-47a5-8868-aa8c90b413a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:55.554876 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdcfe3-9ad5-47a5-8868-aa8c90b413a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:55.555510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbdcfe3-9ad5-47a5-8868-aa8c90b413a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:55.557428 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbdcfe3-9ad5-47a5-8868-aa8c90b413a6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:55.586044 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 992/2011] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:31:55 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:55.590812 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a59b2-068f-4707-8fd0-a7d530476634 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:55.600690 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a59b2-068f-4707-8fd0-a7d530476634 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:55.635462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:55.666645 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fcc411-1c28-4fda-a031-4a4efd90d5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:55.667779 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcc411-1c28-4fda-a031-4a4efd90d5f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:55.668066 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcc411-1c28-4fda-a031-4a4efd90d5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:55.668327 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcc411-1c28-4fda-a031-4a4efd90d5f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:55.668975 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fcc411-1c28-4fda-a031-4a4efd90d5f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:55.671928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fcc411-1c28-4fda-a031-4a4efd90d5f6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:55.699869 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1020/2012] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:31:55 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:55.700842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdcfe3-9ad5-47a5-8868-aa8c90b413a6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:55.711450 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdcfe3-9ad5-47a5-8868-aa8c90b413a6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:55.716811 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4c719-f0f1-442a-b84c-7d61221245db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:55.717428 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4c719-f0f1-442a-b84c-7d61221245db admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:55.717711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4c719-f0f1-442a-b84c-7d61221245db admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:55.717989 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4c719-f0f1-442a-b84c-7d61221245db admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:55.718691 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f4c719-f0f1-442a-b84c-7d61221245db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:55.720288 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f4c719-f0f1-442a-b84c-7d61221245db admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:55.731854 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 993/2013] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:31:55.735712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fcc411-1c28-4fda-a031-4a4efd90d5f6 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:55.745808 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fcc411-1c28-4fda-a031-4a4efd90d5f6 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:55.750497 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa3cd9-7b1d-4481-90f1-84d643787def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:55.751099 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa3cd9-7b1d-4481-90f1-84d643787def admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:55.751385 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa3cd9-7b1d-4481-90f1-84d643787def admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:55.751670 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa3cd9-7b1d-4481-90f1-84d643787def admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:55.752376 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaa3cd9-7b1d-4481-90f1-84d643787def admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:55.754235 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaa3cd9-7b1d-4481-90f1-84d643787def admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:55.764256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1021/2014] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:55.768418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4c719-f0f1-442a-b84c-7d61221245db admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:55.779079 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4c719-f0f1-442a-b84c-7d61221245db admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:55.783703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae4942-b175-44de-9e1e-611335a038cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:55.784349 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae4942-b175-44de-9e1e-611335a038cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:55.784637 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae4942-b175-44de-9e1e-611335a038cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:55.784916 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae4942-b175-44de-9e1e-611335a038cc admin admin] PATH_INFO: `/v3/projects/ef9472793c464d45a2792fe06666fe47/users/167754feaf8b44e9bd9489779cd8912e/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:55.800129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae4942-b175-44de-9e1e-611335a038cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef9472793c464d45a2792fe06666fe47, user_id=167754feaf8b44e9bd9489779cd8912e, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:55.802973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae4942-b175-44de-9e1e-611335a038cc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:55.816266 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 994/2015] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:55 2022] PUT /identity/v3/projects/ef9472793c464d45a2792fe06666fe47/users/167754feaf8b44e9bd9489779cd8912e/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:55.827762 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b4108-24a2-4331-ad22-2878e86a21e2 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:55.828051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b4108-24a2-4331-ad22-2878e86a21e2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:55.828327 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b4108-24a2-4331-ad22-2878e86a21e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:55.857700 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2d9b4108-24a2-4331-ad22-2878e86a21e2 None None] MFA Rules not processed for user `167754feaf8b44e9bd9489779cd8912e`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:55.859022 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2d9b4108-24a2-4331-ad22-2878e86a21e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:55.922929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1022/2016] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:31:55.927569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae4942-b175-44de-9e1e-611335a038cc admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:55.945933 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:55.994531 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae4942-b175-44de-9e1e-611335a038cc admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:55.999351 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed329a15-ad30-4cf6-8fee-bddecddcff98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:55.999962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed329a15-ad30-4cf6-8fee-bddecddcff98 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:56.000246 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed329a15-ad30-4cf6-8fee-bddecddcff98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:56.000521 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed329a15-ad30-4cf6-8fee-bddecddcff98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:56.001229 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed329a15-ad30-4cf6-8fee-bddecddcff98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:56.003098 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed329a15-ad30-4cf6-8fee-bddecddcff98 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:56.032235 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 995/2017] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:31:55 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:31:56.036638 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b4108-24a2-4331-ad22-2878e86a21e2 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:56.053834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b4108-24a2-4331-ad22-2878e86a21e2 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:56.115000 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc809e54-2759-4e7f-89a6-efc2671edf92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:56.115609 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc809e54-2759-4e7f-89a6-efc2671edf92 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:56.115910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc809e54-2759-4e7f-89a6-efc2671edf92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:56.116161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc809e54-2759-4e7f-89a6-efc2671edf92 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:56.116800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc809e54-2759-4e7f-89a6-efc2671edf92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:56.118736 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc809e54-2759-4e7f-89a6-efc2671edf92 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:56.141250 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1023/2018] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:31:56 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:31:56.145249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed329a15-ad30-4cf6-8fee-bddecddcff98 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:56.156275 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed329a15-ad30-4cf6-8fee-bddecddcff98 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:56.161161 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44ce13-66da-4505-8e34-e1d394c21354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:56.161760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44ce13-66da-4505-8e34-e1d394c21354 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:56.162039 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44ce13-66da-4505-8e34-e1d394c21354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:56.162356 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44ce13-66da-4505-8e34-e1d394c21354 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:56.163107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44ce13-66da-4505-8e34-e1d394c21354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:56.164702 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44ce13-66da-4505-8e34-e1d394c21354 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:56.175252 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 996/2019] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:56.179491 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc809e54-2759-4e7f-89a6-efc2671edf92 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:56.189719 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc809e54-2759-4e7f-89a6-efc2671edf92 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:56.194353 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44fefb-5705-4f5a-9724-8e2822445df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:56.194959 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44fefb-5705-4f5a-9724-8e2822445df6 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:56.195245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44fefb-5705-4f5a-9724-8e2822445df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:56.195518 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44fefb-5705-4f5a-9724-8e2822445df6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:56.196166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44fefb-5705-4f5a-9724-8e2822445df6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:56.198281 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44fefb-5705-4f5a-9724-8e2822445df6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:56.211724 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1024/2020] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:56.212922 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44ce13-66da-4505-8e34-e1d394c21354 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:56.223535 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44ce13-66da-4505-8e34-e1d394c21354 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:56.229165 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb458f4-8119-4b80-b8ab-1cc5c30669ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:56.229840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb458f4-8119-4b80-b8ab-1cc5c30669ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:56.230155 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb458f4-8119-4b80-b8ab-1cc5c30669ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:56.230515 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb458f4-8119-4b80-b8ab-1cc5c30669ed admin admin] PATH_INFO: `/v3/projects/4cb995146d1f423ba76419392807ae64/users/24427ffaf9844dbea3c4e1f6efed7ea9/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:56.245671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb458f4-8119-4b80-b8ab-1cc5c30669ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cb995146d1f423ba76419392807ae64, user_id=24427ffaf9844dbea3c4e1f6efed7ea9, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:56.248590 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb458f4-8119-4b80-b8ab-1cc5c30669ed admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:56.263562 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 997/2021] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:56 2022] PUT /identity/v3/projects/4cb995146d1f423ba76419392807ae64/users/24427ffaf9844dbea3c4e1f6efed7ea9/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:56.273619 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fdfb8-1def-4155-ad87-6abacf150b5a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:56.274041 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fdfb8-1def-4155-ad87-6abacf150b5a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:56.274411 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fdfb8-1def-4155-ad87-6abacf150b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:56.303785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e08fdfb8-1def-4155-ad87-6abacf150b5a None None] MFA Rules not processed for user `24427ffaf9844dbea3c4e1f6efed7ea9`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:56.304929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e08fdfb8-1def-4155-ad87-6abacf150b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:56.317533 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:56.378051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1025/2022] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:31:56.382609 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb458f4-8119-4b80-b8ab-1cc5c30669ed admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:56.451596 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb458f4-8119-4b80-b8ab-1cc5c30669ed admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:56.456824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b231a-da53-489f-aab9-78e360a3e376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:56.457627 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b231a-da53-489f-aab9-78e360a3e376 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:56.457736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b231a-da53-489f-aab9-78e360a3e376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:56.457979 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b231a-da53-489f-aab9-78e360a3e376 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:56.458658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b231a-da53-489f-aab9-78e360a3e376 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:56.460550 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b231a-da53-489f-aab9-78e360a3e376 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:56.488829 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 998/2023] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:31:56 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:31:56.492845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08fdfb8-1def-4155-ad87-6abacf150b5a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:56.504010 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08fdfb8-1def-4155-ad87-6abacf150b5a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:56.562085 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f84bba-396e-40f4-92fc-8ce1d0b45e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:56.562623 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f84bba-396e-40f4-92fc-8ce1d0b45e23 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:56.562907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f84bba-396e-40f4-92fc-8ce1d0b45e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:56.563172 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f84bba-396e-40f4-92fc-8ce1d0b45e23 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:56.563854 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f84bba-396e-40f4-92fc-8ce1d0b45e23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:56.565768 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f84bba-396e-40f4-92fc-8ce1d0b45e23 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:56.589905 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1026/2024] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:31:56 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:31:56.593583 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b231a-da53-489f-aab9-78e360a3e376 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:56.603883 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b231a-da53-489f-aab9-78e360a3e376 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:56.609008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e9206f-8559-4695-9d89-82f9b98171cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:56.609607 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9206f-8559-4695-9d89-82f9b98171cb admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:56.609892 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9206f-8559-4695-9d89-82f9b98171cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:56.610170 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9206f-8559-4695-9d89-82f9b98171cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:56.610870 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e9206f-8559-4695-9d89-82f9b98171cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:56.612475 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e9206f-8559-4695-9d89-82f9b98171cb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:56.622855 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 999/2025] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:56.626818 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f84bba-396e-40f4-92fc-8ce1d0b45e23 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:56.637255 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f84bba-396e-40f4-92fc-8ce1d0b45e23 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:56.641948 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094722f-00e2-484e-98c5-c71a968aee0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:56.642714 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094722f-00e2-484e-98c5-c71a968aee0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:56.642999 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094722f-00e2-484e-98c5-c71a968aee0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:56.643276 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094722f-00e2-484e-98c5-c71a968aee0d admin admin] PATH_INFO: `/v3/projects/69aaea70ba454f02ac6202ab45dbd57f/users/cc7b83cefc2243668015393da82b286b/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:56.658301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2094722f-00e2-484e-98c5-c71a968aee0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=69aaea70ba454f02ac6202ab45dbd57f, user_id=cc7b83cefc2243668015393da82b286b, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:56.661241 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2094722f-00e2-484e-98c5-c71a968aee0d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:56.675131 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1027/2026] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:56 2022] PUT /identity/v3/projects/69aaea70ba454f02ac6202ab45dbd57f/users/cc7b83cefc2243668015393da82b286b/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:56.679313 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e9206f-8559-4695-9d89-82f9b98171cb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:56.689409 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e9206f-8559-4695-9d89-82f9b98171cb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:56.727274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:56.747942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a3489-f206-4ab1-ba89-7dc8d2c6164a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:56.748588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a3489-f206-4ab1-ba89-7dc8d2c6164a admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:56.748881 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a3489-f206-4ab1-ba89-7dc8d2c6164a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:56.749159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a3489-f206-4ab1-ba89-7dc8d2c6164a admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:56.750062 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a3489-f206-4ab1-ba89-7dc8d2c6164a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:56.751736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a3489-f206-4ab1-ba89-7dc8d2c6164a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:56.762253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1000/2027] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:31:56.766492 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094722f-00e2-484e-98c5-c71a968aee0d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:56.777582 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094722f-00e2-484e-98c5-c71a968aee0d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:56.782356 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8889ab-c7bb-470a-8ede-3fc15076991a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:56.783085 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8889ab-c7bb-470a-8ede-3fc15076991a admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:56.783419 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8889ab-c7bb-470a-8ede-3fc15076991a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:56.783755 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8889ab-c7bb-470a-8ede-3fc15076991a admin admin] PATH_INFO: `/v3/projects/69aaea70ba454f02ac6202ab45dbd57f/users/cc7b83cefc2243668015393da82b286b/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:56.785967 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8889ab-c7bb-470a-8ede-3fc15076991a admin admin] RBAC: Authorizing `identity:create_grant(project_id=69aaea70ba454f02ac6202ab45dbd57f, user_id=cc7b83cefc2243668015393da82b286b, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:56.788688 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8889ab-c7bb-470a-8ede-3fc15076991a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:56.799230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1028/2028] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:31:56 2022] PUT /identity/v3/projects/69aaea70ba454f02ac6202ab45dbd57f/users/cc7b83cefc2243668015393da82b286b/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:31:56.810327 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2dca6-338b-4fe4-97cc-acd54d8e259f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:56.810617 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2dca6-338b-4fe4-97cc-acd54d8e259f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:56.810893 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2dca6-338b-4fe4-97cc-acd54d8e259f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:56.840611 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c9b2dca6-338b-4fe4-97cc-acd54d8e259f None None] MFA Rules not processed for user `cc7b83cefc2243668015393da82b286b`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:56.841866 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c9b2dca6-338b-4fe4-97cc-acd54d8e259f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:56.931341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1001/2029] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:31:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:31:56.944105 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8889ab-c7bb-470a-8ede-3fc15076991a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:56.946098 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3a8889ab-c7bb-470a-8ede-3fc15076991a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:57.013706 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8889ab-c7bb-470a-8ede-3fc15076991a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:57.017822 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7505682d-5353-46c4-afa6-35cb2d090e9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:57.018511 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7505682d-5353-46c4-afa6-35cb2d090e9d service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:57.018790 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7505682d-5353-46c4-afa6-35cb2d090e9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:57.019089 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7505682d-5353-46c4-afa6-35cb2d090e9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:57.038085 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:57.085724 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7505682d-5353-46c4-afa6-35cb2d090e9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:57.087204 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7505682d-5353-46c4-afa6-35cb2d090e9d service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:57.096609 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1029/2030] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:31:59.709663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2dca6-338b-4fe4-97cc-acd54d8e259f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:59.721648 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2dca6-338b-4fe4-97cc-acd54d8e259f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:59.726262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41382603-2d84-4ddf-8753-117ee46a0049 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:59.726967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41382603-2d84-4ddf-8753-117ee46a0049 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:59.727291 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41382603-2d84-4ddf-8753-117ee46a0049 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:59.727652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41382603-2d84-4ddf-8753-117ee46a0049 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:59.788263 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41382603-2d84-4ddf-8753-117ee46a0049 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:59.789747 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41382603-2d84-4ddf-8753-117ee46a0049 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:59.799250 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1002/2031] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:04.452577 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7505682d-5353-46c4-afa6-35cb2d090e9d service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:04.461429 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7505682d-5353-46c4-afa6-35cb2d090e9d service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:04.465715 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8b3a9-c1ea-427c-b420-fdacd8f645e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:04.466477 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8b3a9-c1ea-427c-b420-fdacd8f645e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:04.466825 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8b3a9-c1ea-427c-b420-fdacd8f645e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:04.467170 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8b3a9-c1ea-427c-b420-fdacd8f645e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:04.476513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b8b3a9-c1ea-427c-b420-fdacd8f645e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:04.478022 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b8b3a9-c1ea-427c-b420-fdacd8f645e6 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:04.488508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1030/2032] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:05.740311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41382603-2d84-4ddf-8753-117ee46a0049 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:05.743618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-41382603-2d84-4ddf-8753-117ee46a0049 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:05.825521 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41382603-2d84-4ddf-8753-117ee46a0049 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:05.833140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8f95d-b129-4743-b204-8fc0cd119dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:05.833788 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8f95d-b129-4743-b204-8fc0cd119dfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:05.834076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8f95d-b129-4743-b204-8fc0cd119dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:05.834391 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8f95d-b129-4743-b204-8fc0cd119dfc admin admin] PATH_INFO: `/v3/users/cb770cf7b9aa4968bd618a88f3a6af7b` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:05.835541 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8f95d-b129-4743-b204-8fc0cd119dfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb770cf7b9aa4968bd618a88f3a6af7b)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:05.837506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8f95d-b129-4743-b204-8fc0cd119dfc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:05.839426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:05.906469 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-25e8f95d-b129-4743-b204-8fc0cd119dfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb770cf7b9aa4968bd618a88f3a6af7b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:05.915409 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-25e8f95d-b129-4743-b204-8fc0cd119dfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb770cf7b9aa4968bd618a88f3a6af7b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:05.917196 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-25e8f95d-b129-4743-b204-8fc0cd119dfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb770cf7b9aa4968bd618a88f3a6af7b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:05.927482 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-25e8f95d-b129-4743-b204-8fc0cd119dfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb770cf7b9aa4968bd618a88f3a6af7b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:05.946772 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-25e8f95d-b129-4743-b204-8fc0cd119dfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb770cf7b9aa4968bd618a88f3a6af7b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:05.955747 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1003/2033] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:32:05 2022] DELETE /identity/v3/users/cb770cf7b9aa4968bd618a88f3a6af7b => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:05.964810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d31a8-9133-4c7c-b74b-65dbcc59d805 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:05.965466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d31a8-9133-4c7c-b74b-65dbcc59d805 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:05.965769 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d31a8-9133-4c7c-b74b-65dbcc59d805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:05.993053 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-432d31a8-9133-4c7c-b74b-65dbcc59d805 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:05.994464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-432d31a8-9133-4c7c-b74b-65dbcc59d805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:06.067802 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1031/2034] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:32:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:06.078035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8f95d-b129-4743-b204-8fc0cd119dfc admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:06.079455 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-25e8f95d-b129-4743-b204-8fc0cd119dfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:06.136552 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8f95d-b129-4743-b204-8fc0cd119dfc admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:06.140154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd114e5-7aa7-433d-8751-d291f7928ec6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:06.140926 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd114e5-7aa7-433d-8751-d291f7928ec6 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:06.141215 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd114e5-7aa7-433d-8751-d291f7928ec6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:06.141486 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd114e5-7aa7-433d-8751-d291f7928ec6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:06.150737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd114e5-7aa7-433d-8751-d291f7928ec6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:06.152165 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd114e5-7aa7-433d-8751-d291f7928ec6 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:06.162130 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1004/2035] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:06.441420 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d31a8-9133-4c7c-b74b-65dbcc59d805 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:06.443554 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-432d31a8-9133-4c7c-b74b-65dbcc59d805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:06.484227 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d31a8-9133-4c7c-b74b-65dbcc59d805 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:06.491285 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d9cea-a36d-4cd1-bae5-9e300adfa73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:06.492207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d9cea-a36d-4cd1-bae5-9e300adfa73a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:06.492621 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d9cea-a36d-4cd1-bae5-9e300adfa73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:06.493036 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d9cea-a36d-4cd1-bae5-9e300adfa73a admin admin] PATH_INFO: `/v3/projects/f643701d94cf45b98130084f478e50b7` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:06.495521 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d9cea-a36d-4cd1-bae5-9e300adfa73a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f643701d94cf45b98130084f478e50b7)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:06.498029 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d9cea-a36d-4cd1-bae5-9e300adfa73a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:06.544898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e53d9cea-a36d-4cd1-bae5-9e300adfa73a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f643701d94cf45b98130084f478e50b7', 'request_id': 'req-e53d9cea-a36d-4cd1-bae5-9e300adfa73a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:06.550977 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e53d9cea-a36d-4cd1-bae5-9e300adfa73a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f643701d94cf45b98130084f478e50b7', 'request_id': 'req-e53d9cea-a36d-4cd1-bae5-9e300adfa73a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:06.556398 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e53d9cea-a36d-4cd1-bae5-9e300adfa73a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f643701d94cf45b98130084f478e50b7', 'request_id': 'req-e53d9cea-a36d-4cd1-bae5-9e300adfa73a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:06.569769 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e53d9cea-a36d-4cd1-bae5-9e300adfa73a admin admin] The token cache is being invalidate because project f643701d94cf45b98130084f478e50b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:32:06.570001 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e53d9cea-a36d-4cd1-bae5-9e300adfa73a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:06.570653 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e53d9cea-a36d-4cd1-bae5-9e300adfa73a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:06.572115 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1032/2036] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:32:06 2022] DELETE /identity/v3/projects/f643701d94cf45b98130084f478e50b7 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:06.576211 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd114e5-7aa7-433d-8751-d291f7928ec6 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:06.585708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd114e5-7aa7-433d-8751-d291f7928ec6 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:06.641239 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be70142-c8a7-4c51-b930-ab59d1c552da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:06.641860 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be70142-c8a7-4c51-b930-ab59d1c552da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:06.642143 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be70142-c8a7-4c51-b930-ab59d1c552da admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:06.642449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be70142-c8a7-4c51-b930-ab59d1c552da admin admin] PATH_INFO: `/v3/users/99f95a35519243faaf4c3aeeba709578` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:06.643583 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be70142-c8a7-4c51-b930-ab59d1c552da admin admin] RBAC: Authorizing `identity:delete_user(user_id=99f95a35519243faaf4c3aeeba709578)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:06.645482 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be70142-c8a7-4c51-b930-ab59d1c552da admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:06.667768 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:06.691902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4be70142-c8a7-4c51-b930-ab59d1c552da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99f95a35519243faaf4c3aeeba709578'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:06.698683 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4be70142-c8a7-4c51-b930-ab59d1c552da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99f95a35519243faaf4c3aeeba709578'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:06.699333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4be70142-c8a7-4c51-b930-ab59d1c552da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99f95a35519243faaf4c3aeeba709578'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:06.706373 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4be70142-c8a7-4c51-b930-ab59d1c552da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99f95a35519243faaf4c3aeeba709578'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:06.718334 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4be70142-c8a7-4c51-b930-ab59d1c552da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99f95a35519243faaf4c3aeeba709578'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:06.724137 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1005/2037] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:32:06 2022] DELETE /identity/v3/users/99f95a35519243faaf4c3aeeba709578 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:07.017804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d9cea-a36d-4cd1-bae5-9e300adfa73a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:07.019348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e53d9cea-a36d-4cd1-bae5-9e300adfa73a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:07.034735 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d9cea-a36d-4cd1-bae5-9e300adfa73a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:07.068653 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:07.102889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f943e-d111-479a-9741-4a99fc9c8e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:07.103477 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f943e-d111-479a-9741-4a99fc9c8e13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:07.104261 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f943e-d111-479a-9741-4a99fc9c8e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:07.104539 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f943e-d111-479a-9741-4a99fc9c8e13 admin admin] PATH_INFO: `/v3/projects/966114545c704a9aa6815cbc2a05db31` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:07.105681 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f943e-d111-479a-9741-4a99fc9c8e13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=966114545c704a9aa6815cbc2a05db31)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:07.107634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f943e-d111-479a-9741-4a99fc9c8e13 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:07.168484 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-305f943e-d111-479a-9741-4a99fc9c8e13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '966114545c704a9aa6815cbc2a05db31', 'request_id': 'req-305f943e-d111-479a-9741-4a99fc9c8e13'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:07.175787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-305f943e-d111-479a-9741-4a99fc9c8e13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '966114545c704a9aa6815cbc2a05db31', 'request_id': 'req-305f943e-d111-479a-9741-4a99fc9c8e13'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:07.183388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-305f943e-d111-479a-9741-4a99fc9c8e13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '966114545c704a9aa6815cbc2a05db31', 'request_id': 'req-305f943e-d111-479a-9741-4a99fc9c8e13'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:07.197940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-305f943e-d111-479a-9741-4a99fc9c8e13 admin admin] The token cache is being invalidate because project 966114545c704a9aa6815cbc2a05db31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:32:07.198287 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-305f943e-d111-479a-9741-4a99fc9c8e13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:07.198978 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-305f943e-d111-479a-9741-4a99fc9c8e13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:07.201005 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1033/2038] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:32:07 2022] DELETE /identity/v3/projects/966114545c704a9aa6815cbc2a05db31 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:07.206034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be70142-c8a7-4c51-b930-ab59d1c552da admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:07.207708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4be70142-c8a7-4c51-b930-ab59d1c552da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:07.221477 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be70142-c8a7-4c51-b930-ab59d1c552da admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:07.279758 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419d012-38e4-42b5-9141-8fdbf2f58b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:07.280380 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419d012-38e4-42b5-9141-8fdbf2f58b52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:07.280662 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419d012-38e4-42b5-9141-8fdbf2f58b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:07.280934 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419d012-38e4-42b5-9141-8fdbf2f58b52 admin admin] PATH_INFO: `/v3/users/c152e984d9424a2fb8764c35177410ac` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:07.282037 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7419d012-38e4-42b5-9141-8fdbf2f58b52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c152e984d9424a2fb8764c35177410ac)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:07.283975 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7419d012-38e4-42b5-9141-8fdbf2f58b52 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:07.344138 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7419d012-38e4-42b5-9141-8fdbf2f58b52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c152e984d9424a2fb8764c35177410ac'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:07.344138 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7419d012-38e4-42b5-9141-8fdbf2f58b52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c152e984d9424a2fb8764c35177410ac'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:07.344138 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7419d012-38e4-42b5-9141-8fdbf2f58b52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c152e984d9424a2fb8764c35177410ac'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:07.350768 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7419d012-38e4-42b5-9141-8fdbf2f58b52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c152e984d9424a2fb8764c35177410ac'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:07.363713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7419d012-38e4-42b5-9141-8fdbf2f58b52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c152e984d9424a2fb8764c35177410ac'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:07.369764 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1006/2039] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:32:07 2022] DELETE /identity/v3/users/c152e984d9424a2fb8764c35177410ac => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:07.677218 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f943e-d111-479a-9741-4a99fc9c8e13 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:07.678854 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-305f943e-d111-479a-9741-4a99fc9c8e13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:07.692472 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f943e-d111-479a-9741-4a99fc9c8e13 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:07.759548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b943f8f-ee92-4a70-af25-9ca4fd78951c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:07.760304 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b943f8f-ee92-4a70-af25-9ca4fd78951c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:07.760628 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b943f8f-ee92-4a70-af25-9ca4fd78951c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:07.761389 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b943f8f-ee92-4a70-af25-9ca4fd78951c admin admin] PATH_INFO: `/v3/projects/d686e2d585f849dc9832831136dfa705` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:07.762563 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b943f8f-ee92-4a70-af25-9ca4fd78951c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d686e2d585f849dc9832831136dfa705)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:07.764495 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b943f8f-ee92-4a70-af25-9ca4fd78951c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:07.813478 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8b943f8f-ee92-4a70-af25-9ca4fd78951c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd686e2d585f849dc9832831136dfa705', 'request_id': 'req-8b943f8f-ee92-4a70-af25-9ca4fd78951c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:07.819966 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8b943f8f-ee92-4a70-af25-9ca4fd78951c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd686e2d585f849dc9832831136dfa705', 'request_id': 'req-8b943f8f-ee92-4a70-af25-9ca4fd78951c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:07.825299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:07.826946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8b943f8f-ee92-4a70-af25-9ca4fd78951c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd686e2d585f849dc9832831136dfa705', 'request_id': 'req-8b943f8f-ee92-4a70-af25-9ca4fd78951c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:07.841853 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8b943f8f-ee92-4a70-af25-9ca4fd78951c admin admin] The token cache is being invalidate because project d686e2d585f849dc9832831136dfa705 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:32:07.842157 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8b943f8f-ee92-4a70-af25-9ca4fd78951c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:07.842888 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8b943f8f-ee92-4a70-af25-9ca4fd78951c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:07.844376 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1034/2040] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:32:07 2022] DELETE /identity/v3/projects/d686e2d585f849dc9832831136dfa705 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:07.850405 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b91df-d1c1-4372-9613-a39400f1c172 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:07.850695 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b91df-d1c1-4372-9613-a39400f1c172 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:07.850970 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b91df-d1c1-4372-9613-a39400f1c172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:07.871330 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-291b91df-d1c1-4372-9613-a39400f1c172 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:07.872531 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-291b91df-d1c1-4372-9613-a39400f1c172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:07.885998 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:07.934896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1007/2041] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:07.943395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca489353-32b1-4c45-84a7-f288e7bb6224 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:07.943683 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca489353-32b1-4c45-84a7-f288e7bb6224 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:07.943963 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca489353-32b1-4c45-84a7-f288e7bb6224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:07.963810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ca489353-32b1-4c45-84a7-f288e7bb6224 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:07.965022 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ca489353-32b1-4c45-84a7-f288e7bb6224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:07.972929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1035/2042] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:07.977309 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b91df-d1c1-4372-9613-a39400f1c172 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:07.987817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b91df-d1c1-4372-9613-a39400f1c172 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:07.992394 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215aebc-d376-4155-a15b-73dad215ccdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:07.993005 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215aebc-d376-4155-a15b-73dad215ccdc admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:07.993290 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215aebc-d376-4155-a15b-73dad215ccdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:07.993562 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215aebc-d376-4155-a15b-73dad215ccdc admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:07.994917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3215aebc-d376-4155-a15b-73dad215ccdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:32:07.995225 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3215aebc-d376-4155-a15b-73dad215ccdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:07.996830 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3215aebc-d376-4155-a15b-73dad215ccdc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:08.012198 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1008/2043] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:32:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:32:08.016483 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca489353-32b1-4c45-84a7-f288e7bb6224 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:08.026602 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca489353-32b1-4c45-84a7-f288e7bb6224 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:08.031806 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adbe6db-6568-41cd-b053-f330894495d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:08.032427 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbe6db-6568-41cd-b053-f330894495d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:08.032713 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbe6db-6568-41cd-b053-f330894495d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:08.033006 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbe6db-6568-41cd-b053-f330894495d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:08.033641 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adbe6db-6568-41cd-b053-f330894495d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:08.035862 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adbe6db-6568-41cd-b053-f330894495d6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:08.082256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1036/2044] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:32:08 2022] POST /identity/v3/projects => generated 399 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:08.084317 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215aebc-d376-4155-a15b-73dad215ccdc admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:08.096249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215aebc-d376-4155-a15b-73dad215ccdc admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:08.173088 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb787f24-5ea5-41f7-b4ce-ca192e9b7e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:08.173088 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb787f24-5ea5-41f7-b4ce-ca192e9b7e6e admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:08.173088 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb787f24-5ea5-41f7-b4ce-ca192e9b7e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:08.173088 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb787f24-5ea5-41f7-b4ce-ca192e9b7e6e admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:08.173409 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb787f24-5ea5-41f7-b4ce-ca192e9b7e6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:08.173409 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb787f24-5ea5-41f7-b4ce-ca192e9b7e6e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:08.202948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1009/2045] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:32:08 2022] POST /identity/v3/users => generated 300 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:32:08.204746 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adbe6db-6568-41cd-b053-f330894495d6 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:08.215736 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adbe6db-6568-41cd-b053-f330894495d6 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:08.220366 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58e446-dd27-4551-a4f8-a451a00e3c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:08.221032 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58e446-dd27-4551-a4f8-a451a00e3c2e admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:08.221311 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58e446-dd27-4551-a4f8-a451a00e3c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:08.221579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58e446-dd27-4551-a4f8-a451a00e3c2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:08.222247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d58e446-dd27-4551-a4f8-a451a00e3c2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:08.223829 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d58e446-dd27-4551-a4f8-a451a00e3c2e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:08.234553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1037/2046] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:08.238965 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb787f24-5ea5-41f7-b4ce-ca192e9b7e6e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:08.256248 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb787f24-5ea5-41f7-b4ce-ca192e9b7e6e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:08.262960 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a446b5-4acd-4c31-b7b4-d374ac26b781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:08.263578 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a446b5-4acd-4c31-b7b4-d374ac26b781 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:08.263957 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a446b5-4acd-4c31-b7b4-d374ac26b781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:08.264124 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a446b5-4acd-4c31-b7b4-d374ac26b781 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:08.264782 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a446b5-4acd-4c31-b7b4-d374ac26b781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:08.266417 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a446b5-4acd-4c31-b7b4-d374ac26b781 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:08.280580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1010/2047] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:08.284966 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58e446-dd27-4551-a4f8-a451a00e3c2e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:08.306644 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58e446-dd27-4551-a4f8-a451a00e3c2e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:08.306644 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f78bf3-2342-48da-88a6-c7cc0af9cfd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:08.306644 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f78bf3-2342-48da-88a6-c7cc0af9cfd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:08.306644 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f78bf3-2342-48da-88a6-c7cc0af9cfd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:08.306946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f78bf3-2342-48da-88a6-c7cc0af9cfd4 admin admin] PATH_INFO: `/v3/projects/cfcb68292ea0475e869133e985466316/users/2b15e963e007424d8ce9da80a7a96b87/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:08.335641 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f78bf3-2342-48da-88a6-c7cc0af9cfd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfcb68292ea0475e869133e985466316, user_id=2b15e963e007424d8ce9da80a7a96b87, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:08.339891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f78bf3-2342-48da-88a6-c7cc0af9cfd4 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:08.376077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1038/2048] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:32:08 2022] PUT /identity/v3/projects/cfcb68292ea0475e869133e985466316/users/2b15e963e007424d8ce9da80a7a96b87/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:08.377052 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8b0b0-1fbb-478d-849f-49dc45489966 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:08.377348 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8b0b0-1fbb-478d-849f-49dc45489966 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:08.377630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8b0b0-1fbb-478d-849f-49dc45489966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:08.442012 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-23f8b0b0-1fbb-478d-849f-49dc45489966 None None] MFA Rules not processed for user `2b15e963e007424d8ce9da80a7a96b87`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:08.443314 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-23f8b0b0-1fbb-478d-849f-49dc45489966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:08.586455 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1011/2049] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:32:08 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:08.599135 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f78bf3-2342-48da-88a6-c7cc0af9cfd4 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:08.601132 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-17f78bf3-2342-48da-88a6-c7cc0af9cfd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:08.657980 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f78bf3-2342-48da-88a6-c7cc0af9cfd4 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:08.661981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475ff7c-56d3-4b2d-902a-6d7aec48f7bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:08.662608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475ff7c-56d3-4b2d-902a-6d7aec48f7bd service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:08.662885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475ff7c-56d3-4b2d-902a-6d7aec48f7bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:08.663153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475ff7c-56d3-4b2d-902a-6d7aec48f7bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:08.672611 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5475ff7c-56d3-4b2d-902a-6d7aec48f7bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:08.674092 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5475ff7c-56d3-4b2d-902a-6d7aec48f7bd service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:08.683217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1039/2050] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:08 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:09.842344 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8b0b0-1fbb-478d-849f-49dc45489966 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:09.844121 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-23f8b0b0-1fbb-478d-849f-49dc45489966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:09.869123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:09.932845 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8b0b0-1fbb-478d-849f-49dc45489966 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:09.937662 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754ae86-a73d-4fc3-a375-38d627dbdbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:09.938524 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754ae86-a73d-4fc3-a375-38d627dbdbdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:09.939272 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754ae86-a73d-4fc3-a375-38d627dbdbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:09.939272 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754ae86-a73d-4fc3-a375-38d627dbdbdf admin admin] PATH_INFO: `/v3/users/167754feaf8b44e9bd9489779cd8912e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:09.940548 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5754ae86-a73d-4fc3-a375-38d627dbdbdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=167754feaf8b44e9bd9489779cd8912e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:09.942643 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5754ae86-a73d-4fc3-a375-38d627dbdbdf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:10.012193 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5754ae86-a73d-4fc3-a375-38d627dbdbdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '167754feaf8b44e9bd9489779cd8912e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:10.022168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5754ae86-a73d-4fc3-a375-38d627dbdbdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '167754feaf8b44e9bd9489779cd8912e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:10.022974 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5754ae86-a73d-4fc3-a375-38d627dbdbdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '167754feaf8b44e9bd9489779cd8912e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:10.033179 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5754ae86-a73d-4fc3-a375-38d627dbdbdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '167754feaf8b44e9bd9489779cd8912e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:10.048846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5754ae86-a73d-4fc3-a375-38d627dbdbdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '167754feaf8b44e9bd9489779cd8912e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:10.055902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1012/2051] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:32:09 2022] DELETE /identity/v3/users/167754feaf8b44e9bd9489779cd8912e => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:10.060501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347e668-4a8e-4129-844e-bd4597675eab None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:10.060962 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347e668-4a8e-4129-844e-bd4597675eab None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:10.061045 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347e668-4a8e-4129-844e-bd4597675eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:10.084952 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e347e668-4a8e-4129-844e-bd4597675eab None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:10.086226 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e347e668-4a8e-4129-844e-bd4597675eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:10.152167 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1040/2052] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:10.158916 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754ae86-a73d-4fc3-a375-38d627dbdbdf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:10.160759 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5754ae86-a73d-4fc3-a375-38d627dbdbdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:10.244674 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754ae86-a73d-4fc3-a375-38d627dbdbdf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:10.248840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e32cce4-ca70-4e49-8e75-15f7d8025538 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:10.249437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32cce4-ca70-4e49-8e75-15f7d8025538 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:10.249788 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32cce4-ca70-4e49-8e75-15f7d8025538 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:10.250152 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32cce4-ca70-4e49-8e75-15f7d8025538 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:10.260679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e32cce4-ca70-4e49-8e75-15f7d8025538 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:10.262401 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e32cce4-ca70-4e49-8e75-15f7d8025538 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:10.276697 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1013/2053] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:10.571724 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347e668-4a8e-4129-844e-bd4597675eab None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:10.573205 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e347e668-4a8e-4129-844e-bd4597675eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:10.586517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347e668-4a8e-4129-844e-bd4597675eab None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:10.591626 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74d37b-81e0-45ff-82ac-dd722ef3cbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:10.592942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74d37b-81e0-45ff-82ac-dd722ef3cbcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:10.593233 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74d37b-81e0-45ff-82ac-dd722ef3cbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:10.593855 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74d37b-81e0-45ff-82ac-dd722ef3cbcd admin admin] PATH_INFO: `/v3/projects/ef9472793c464d45a2792fe06666fe47` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:10.595040 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74d37b-81e0-45ff-82ac-dd722ef3cbcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef9472793c464d45a2792fe06666fe47)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:10.597066 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74d37b-81e0-45ff-82ac-dd722ef3cbcd admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:10.622773 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:10.645297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ca74d37b-81e0-45ff-82ac-dd722ef3cbcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef9472793c464d45a2792fe06666fe47', 'request_id': 'req-ca74d37b-81e0-45ff-82ac-dd722ef3cbcd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:10.651868 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ca74d37b-81e0-45ff-82ac-dd722ef3cbcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef9472793c464d45a2792fe06666fe47', 'request_id': 'req-ca74d37b-81e0-45ff-82ac-dd722ef3cbcd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:10.657314 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ca74d37b-81e0-45ff-82ac-dd722ef3cbcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef9472793c464d45a2792fe06666fe47', 'request_id': 'req-ca74d37b-81e0-45ff-82ac-dd722ef3cbcd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:10.670307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ca74d37b-81e0-45ff-82ac-dd722ef3cbcd admin admin] The token cache is being invalidate because project ef9472793c464d45a2792fe06666fe47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:32:10.670594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ca74d37b-81e0-45ff-82ac-dd722ef3cbcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:10.671206 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ca74d37b-81e0-45ff-82ac-dd722ef3cbcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:10.672656 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1041/2054] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:32:10 2022] DELETE /identity/v3/projects/ef9472793c464d45a2792fe06666fe47 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:10.676646 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e32cce4-ca70-4e49-8e75-15f7d8025538 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:10.686305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e32cce4-ca70-4e49-8e75-15f7d8025538 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:10.719876 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:10.743698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b155ab-52d2-4152-b664-2e268a4761c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:10.744325 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b155ab-52d2-4152-b664-2e268a4761c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:10.744604 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b155ab-52d2-4152-b664-2e268a4761c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:10.744874 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b155ab-52d2-4152-b664-2e268a4761c7 admin admin] PATH_INFO: `/v3/users/24427ffaf9844dbea3c4e1f6efed7ea9` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:10.745971 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b155ab-52d2-4152-b664-2e268a4761c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24427ffaf9844dbea3c4e1f6efed7ea9)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:10.747952 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b155ab-52d2-4152-b664-2e268a4761c7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:10.799575 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-97b155ab-52d2-4152-b664-2e268a4761c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24427ffaf9844dbea3c4e1f6efed7ea9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:10.808157 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-97b155ab-52d2-4152-b664-2e268a4761c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24427ffaf9844dbea3c4e1f6efed7ea9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:10.809808 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-97b155ab-52d2-4152-b664-2e268a4761c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24427ffaf9844dbea3c4e1f6efed7ea9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:10.831821 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-97b155ab-52d2-4152-b664-2e268a4761c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24427ffaf9844dbea3c4e1f6efed7ea9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:10.845762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-97b155ab-52d2-4152-b664-2e268a4761c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24427ffaf9844dbea3c4e1f6efed7ea9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:10.851932 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1014/2055] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:32:10 2022] DELETE /identity/v3/users/24427ffaf9844dbea3c4e1f6efed7ea9 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:32:11.224534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74d37b-81e0-45ff-82ac-dd722ef3cbcd admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:11.229043 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ca74d37b-81e0-45ff-82ac-dd722ef3cbcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:11.254335 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74d37b-81e0-45ff-82ac-dd722ef3cbcd admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:11.334279 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fa200-c4d7-43d6-b842-a8af1dcb48dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:11.335036 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fa200-c4d7-43d6-b842-a8af1dcb48dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:11.335512 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fa200-c4d7-43d6-b842-a8af1dcb48dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:11.335852 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fa200-c4d7-43d6-b842-a8af1dcb48dd admin admin] PATH_INFO: `/v3/projects/4cb995146d1f423ba76419392807ae64` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:11.337063 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47fa200-c4d7-43d6-b842-a8af1dcb48dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cb995146d1f423ba76419392807ae64)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:11.339608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47fa200-c4d7-43d6-b842-a8af1dcb48dd admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:11.392020 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e47fa200-c4d7-43d6-b842-a8af1dcb48dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cb995146d1f423ba76419392807ae64', 'request_id': 'req-e47fa200-c4d7-43d6-b842-a8af1dcb48dd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:11.399065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e47fa200-c4d7-43d6-b842-a8af1dcb48dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cb995146d1f423ba76419392807ae64', 'request_id': 'req-e47fa200-c4d7-43d6-b842-a8af1dcb48dd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:11.405315 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e47fa200-c4d7-43d6-b842-a8af1dcb48dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cb995146d1f423ba76419392807ae64', 'request_id': 'req-e47fa200-c4d7-43d6-b842-a8af1dcb48dd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:11.421163 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e47fa200-c4d7-43d6-b842-a8af1dcb48dd admin admin] The token cache is being invalidate because project 4cb995146d1f423ba76419392807ae64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:32:11.421688 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e47fa200-c4d7-43d6-b842-a8af1dcb48dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:11.422368 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e47fa200-c4d7-43d6-b842-a8af1dcb48dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:11.424056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1042/2056] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:32:11 2022] DELETE /identity/v3/projects/4cb995146d1f423ba76419392807ae64 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:11.428275 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b155ab-52d2-4152-b664-2e268a4761c7 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:11.429907 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-97b155ab-52d2-4152-b664-2e268a4761c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:11.443519 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b155ab-52d2-4152-b664-2e268a4761c7 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:11.516452 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9709e0d-e611-4614-8ed9-c56af05df2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:11.517361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9709e0d-e611-4614-8ed9-c56af05df2bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:11.517777 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9709e0d-e611-4614-8ed9-c56af05df2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:11.518100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9709e0d-e611-4614-8ed9-c56af05df2bd admin admin] PATH_INFO: `/v3/users/cc7b83cefc2243668015393da82b286b` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:11.519428 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9709e0d-e611-4614-8ed9-c56af05df2bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc7b83cefc2243668015393da82b286b)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:11.521396 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9709e0d-e611-4614-8ed9-c56af05df2bd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:11.574790 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e9709e0d-e611-4614-8ed9-c56af05df2bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc7b83cefc2243668015393da82b286b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:11.582176 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e9709e0d-e611-4614-8ed9-c56af05df2bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc7b83cefc2243668015393da82b286b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:11.582894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e9709e0d-e611-4614-8ed9-c56af05df2bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc7b83cefc2243668015393da82b286b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:11.585977 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:11.592421 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e9709e0d-e611-4614-8ed9-c56af05df2bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc7b83cefc2243668015393da82b286b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:11.606279 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e9709e0d-e611-4614-8ed9-c56af05df2bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc7b83cefc2243668015393da82b286b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:11.612401 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1015/2057] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:32:11 2022] DELETE /identity/v3/users/cc7b83cefc2243668015393da82b286b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:11.934625 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fa200-c4d7-43d6-b842-a8af1dcb48dd admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:11.936212 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e47fa200-c4d7-43d6-b842-a8af1dcb48dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:11.949680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fa200-c4d7-43d6-b842-a8af1dcb48dd admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:11.999281 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:12.018670 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf4b3c-cb4f-46f7-a3d6-f3e927b4b784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:12.019303 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf4b3c-cb4f-46f7-a3d6-f3e927b4b784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:12.019586 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf4b3c-cb4f-46f7-a3d6-f3e927b4b784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:12.019859 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf4b3c-cb4f-46f7-a3d6-f3e927b4b784 admin admin] PATH_INFO: `/v3/projects/69aaea70ba454f02ac6202ab45dbd57f` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:12.021150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf4b3c-cb4f-46f7-a3d6-f3e927b4b784 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69aaea70ba454f02ac6202ab45dbd57f)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:12.023052 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf4b3c-cb4f-46f7-a3d6-f3e927b4b784 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:12.093204 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d3cf4b3c-cb4f-46f7-a3d6-f3e927b4b784 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69aaea70ba454f02ac6202ab45dbd57f', 'request_id': 'req-d3cf4b3c-cb4f-46f7-a3d6-f3e927b4b784'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:12.101473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d3cf4b3c-cb4f-46f7-a3d6-f3e927b4b784 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69aaea70ba454f02ac6202ab45dbd57f', 'request_id': 'req-d3cf4b3c-cb4f-46f7-a3d6-f3e927b4b784'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:12.108839 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d3cf4b3c-cb4f-46f7-a3d6-f3e927b4b784 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69aaea70ba454f02ac6202ab45dbd57f', 'request_id': 'req-d3cf4b3c-cb4f-46f7-a3d6-f3e927b4b784'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:12.123907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d3cf4b3c-cb4f-46f7-a3d6-f3e927b4b784 admin admin] The token cache is being invalidate because project 69aaea70ba454f02ac6202ab45dbd57f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:32:12.124602 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d3cf4b3c-cb4f-46f7-a3d6-f3e927b4b784 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:12.125482 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d3cf4b3c-cb4f-46f7-a3d6-f3e927b4b784 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:12.127785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1043/2058] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:32:11 2022] DELETE /identity/v3/projects/69aaea70ba454f02ac6202ab45dbd57f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:12.132902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a3d05-7b51-4a38-953a-79f63c37d7ec None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:12.133829 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a3d05-7b51-4a38-953a-79f63c37d7ec None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:12.134123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a3d05-7b51-4a38-953a-79f63c37d7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:12.195146 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7c2a3d05-7b51-4a38-953a-79f63c37d7ec None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:12.196612 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7c2a3d05-7b51-4a38-953a-79f63c37d7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:12.299455 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1016/2059] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:12.307070 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6070bd-f74a-4c98-a14f-895888d25be3 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:12.307929 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6070bd-f74a-4c98-a14f-895888d25be3 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:12.308375 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6070bd-f74a-4c98-a14f-895888d25be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:12.331700 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-0b6070bd-f74a-4c98-a14f-895888d25be3 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:12.333071 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0b6070bd-f74a-4c98-a14f-895888d25be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:12.342788 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1044/2060] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:12.347661 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a3d05-7b51-4a38-953a-79f63c37d7ec None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:12.361663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a3d05-7b51-4a38-953a-79f63c37d7ec None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:12.367519 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a0d01-adc5-49f9-8168-6fcd29685279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:12.368189 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a0d01-adc5-49f9-8168-6fcd29685279 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:12.368515 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a0d01-adc5-49f9-8168-6fcd29685279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:12.368880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a0d01-adc5-49f9-8168-6fcd29685279 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:12.369540 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a0d01-adc5-49f9-8168-6fcd29685279 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:32:12.370341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a0d01-adc5-49f9-8168-6fcd29685279 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:12.371607 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a0d01-adc5-49f9-8168-6fcd29685279 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:12.387554 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1017/2061] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:32:12.391747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6070bd-f74a-4c98-a14f-895888d25be3 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:12.411674 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6070bd-f74a-4c98-a14f-895888d25be3 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:12.421306 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d0c9a-9fe8-4296-8459-5402a7177691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:12.424245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d0c9a-9fe8-4296-8459-5402a7177691 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:12.424683 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d0c9a-9fe8-4296-8459-5402a7177691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:12.425106 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d0c9a-9fe8-4296-8459-5402a7177691 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:12.425928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d0c9a-9fe8-4296-8459-5402a7177691 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:12.429768 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d0c9a-9fe8-4296-8459-5402a7177691 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:12.472218 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1045/2062] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:32:12 2022] POST /identity/v3/projects => generated 391 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:12.476048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a0d01-adc5-49f9-8168-6fcd29685279 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:12.487086 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a0d01-adc5-49f9-8168-6fcd29685279 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:12.571322 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8107d6-af89-483b-a4f4-dd3b28fab4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:12.572470 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8107d6-af89-483b-a4f4-dd3b28fab4f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:12.572750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8107d6-af89-483b-a4f4-dd3b28fab4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:12.573021 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8107d6-af89-483b-a4f4-dd3b28fab4f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:12.573738 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8107d6-af89-483b-a4f4-dd3b28fab4f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:12.575695 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8107d6-af89-483b-a4f4-dd3b28fab4f6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:12.598262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1018/2063] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:32:12 2022] POST /identity/v3/users => generated 296 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:12.602261 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d0c9a-9fe8-4296-8459-5402a7177691 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:12.613026 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d0c9a-9fe8-4296-8459-5402a7177691 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:12.617843 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d26d6-9b2a-4eb6-9f82-7d2f10be8a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:12.618597 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d26d6-9b2a-4eb6-9f82-7d2f10be8a03 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:12.618980 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d26d6-9b2a-4eb6-9f82-7d2f10be8a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:12.619322 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d26d6-9b2a-4eb6-9f82-7d2f10be8a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:12.619996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d26d6-9b2a-4eb6-9f82-7d2f10be8a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:12.621600 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d26d6-9b2a-4eb6-9f82-7d2f10be8a03 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:12.632774 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1046/2064] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:12.637226 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8107d6-af89-483b-a4f4-dd3b28fab4f6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:12.647993 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8107d6-af89-483b-a4f4-dd3b28fab4f6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:12.653924 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d003987-cb09-4ef0-8935-0b5c83878676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:12.654672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d003987-cb09-4ef0-8935-0b5c83878676 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:12.654964 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d003987-cb09-4ef0-8935-0b5c83878676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:12.655317 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d003987-cb09-4ef0-8935-0b5c83878676 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:12.655984 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d003987-cb09-4ef0-8935-0b5c83878676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:12.657614 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d003987-cb09-4ef0-8935-0b5c83878676 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:12.673364 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1019/2065] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:12.674872 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d26d6-9b2a-4eb6-9f82-7d2f10be8a03 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:12.685871 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d26d6-9b2a-4eb6-9f82-7d2f10be8a03 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:12.690723 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6954c61-d22e-4da0-8062-fc2be2b5f0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:12.691401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6954c61-d22e-4da0-8062-fc2be2b5f0ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:12.691685 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6954c61-d22e-4da0-8062-fc2be2b5f0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:12.691991 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6954c61-d22e-4da0-8062-fc2be2b5f0ba admin admin] PATH_INFO: `/v3/projects/59256b7f61424ae4b6038009232caac8/users/c3b955c057614d3fbcc01ecfecdb4931/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:12.708402 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6954c61-d22e-4da0-8062-fc2be2b5f0ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=59256b7f61424ae4b6038009232caac8, user_id=c3b955c057614d3fbcc01ecfecdb4931, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:12.711109 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6954c61-d22e-4da0-8062-fc2be2b5f0ba admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:12.727725 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1047/2066] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:32:12 2022] PUT /identity/v3/projects/59256b7f61424ae4b6038009232caac8/users/c3b955c057614d3fbcc01ecfecdb4931/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:12.738436 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701bb46-9335-4e37-81a5-cf17715fcc8d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:12.738739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701bb46-9335-4e37-81a5-cf17715fcc8d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:12.739024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701bb46-9335-4e37-81a5-cf17715fcc8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:12.769392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6701bb46-9335-4e37-81a5-cf17715fcc8d None None] MFA Rules not processed for user `c3b955c057614d3fbcc01ecfecdb4931`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:12.770659 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6701bb46-9335-4e37-81a5-cf17715fcc8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:12.802999 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:12.840397 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1020/2067] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:32:12 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:12.851152 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6954c61-d22e-4da0-8062-fc2be2b5f0ba admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:12.852777 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d6954c61-d22e-4da0-8062-fc2be2b5f0ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:12.910523 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6954c61-d22e-4da0-8062-fc2be2b5f0ba admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:12.914151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bced2f9-afa6-4509-9452-b7fdc761f40a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:12.914784 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bced2f9-afa6-4509-9452-b7fdc761f40a service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:12.915067 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bced2f9-afa6-4509-9452-b7fdc761f40a service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:12.915344 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bced2f9-afa6-4509-9452-b7fdc761f40a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:12.923802 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bced2f9-afa6-4509-9452-b7fdc761f40a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:12.925227 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bced2f9-afa6-4509-9452-b7fdc761f40a service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:12.934762 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1048/2068] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:12 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:22.836894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=13 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:22.929969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:25.207867 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701bb46-9335-4e37-81a5-cf17715fcc8d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:25.209614 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6701bb46-9335-4e37-81a5-cf17715fcc8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:25.318881 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701bb46-9335-4e37-81a5-cf17715fcc8d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:25.323889 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283c695-380b-4290-8837-1facf7373d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:25.324527 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283c695-380b-4290-8837-1facf7373d1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:25.324806 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283c695-380b-4290-8837-1facf7373d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:25.325084 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283c695-380b-4290-8837-1facf7373d1a admin admin] PATH_INFO: `/v3/users/2b15e963e007424d8ce9da80a7a96b87` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:25.326257 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283c695-380b-4290-8837-1facf7373d1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b15e963e007424d8ce9da80a7a96b87)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:25.328168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283c695-380b-4290-8837-1facf7373d1a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:25.378634 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b283c695-380b-4290-8837-1facf7373d1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b15e963e007424d8ce9da80a7a96b87'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:25.386329 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b283c695-380b-4290-8837-1facf7373d1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b15e963e007424d8ce9da80a7a96b87'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:25.387065 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b283c695-380b-4290-8837-1facf7373d1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b15e963e007424d8ce9da80a7a96b87'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:25.394930 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b283c695-380b-4290-8837-1facf7373d1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b15e963e007424d8ce9da80a7a96b87'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:25.407899 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b283c695-380b-4290-8837-1facf7373d1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b15e963e007424d8ce9da80a7a96b87'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:25.414068 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1021/2069] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:32:25 2022] DELETE /identity/v3/users/2b15e963e007424d8ce9da80a7a96b87 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:25.418944 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f80510-69f0-4a53-ba87-8607ee070802 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:25.419499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f80510-69f0-4a53-ba87-8607ee070802 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:25.419499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f80510-69f0-4a53-ba87-8607ee070802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:25.438790 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-00f80510-69f0-4a53-ba87-8607ee070802 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:25.440052 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-00f80510-69f0-4a53-ba87-8607ee070802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:25.499428 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1049/2070] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:32:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:32:25.509766 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283c695-380b-4290-8837-1facf7373d1a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:25.511177 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b283c695-380b-4290-8837-1facf7373d1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:25.568650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283c695-380b-4290-8837-1facf7373d1a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:25.573806 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d6e91-c7ab-4c02-9487-d025f488fb88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:25.574444 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d6e91-c7ab-4c02-9487-d025f488fb88 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:25.574731 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d6e91-c7ab-4c02-9487-d025f488fb88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:25.575037 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d6e91-c7ab-4c02-9487-d025f488fb88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:25.584570 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d6e91-c7ab-4c02-9487-d025f488fb88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:25.585991 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d6e91-c7ab-4c02-9487-d025f488fb88 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:25.596052 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1022/2071] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:25.877800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f80510-69f0-4a53-ba87-8607ee070802 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:25.879338 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-00f80510-69f0-4a53-ba87-8607ee070802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:25.891962 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f80510-69f0-4a53-ba87-8607ee070802 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:25.896947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04363375-71d6-4543-a650-e15250bcf462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:25.897577 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04363375-71d6-4543-a650-e15250bcf462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:25.897860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04363375-71d6-4543-a650-e15250bcf462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:25.898137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04363375-71d6-4543-a650-e15250bcf462 admin admin] PATH_INFO: `/v3/projects/cfcb68292ea0475e869133e985466316` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:25.899324 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04363375-71d6-4543-a650-e15250bcf462 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfcb68292ea0475e869133e985466316)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:25.901244 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04363375-71d6-4543-a650-e15250bcf462 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:25.948909 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-04363375-71d6-4543-a650-e15250bcf462 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfcb68292ea0475e869133e985466316', 'request_id': 'req-04363375-71d6-4543-a650-e15250bcf462'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:25.954962 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-04363375-71d6-4543-a650-e15250bcf462 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfcb68292ea0475e869133e985466316', 'request_id': 'req-04363375-71d6-4543-a650-e15250bcf462'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:25.961308 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-04363375-71d6-4543-a650-e15250bcf462 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfcb68292ea0475e869133e985466316', 'request_id': 'req-04363375-71d6-4543-a650-e15250bcf462'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:25.972713 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-04363375-71d6-4543-a650-e15250bcf462 admin admin] The token cache is being invalidate because project cfcb68292ea0475e869133e985466316 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:32:25.973002 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-04363375-71d6-4543-a650-e15250bcf462 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:25.973615 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-04363375-71d6-4543-a650-e15250bcf462 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:25.975120 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1050/2072] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:32:25 2022] DELETE /identity/v3/projects/cfcb68292ea0475e869133e985466316 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:35.591624 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:35.969903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:41.524320 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d6e91-c7ab-4c02-9487-d025f488fb88 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:41.525888 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1f3d6e91-c7ab-4c02-9487-d025f488fb88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:41.539326 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d6e91-c7ab-4c02-9487-d025f488fb88 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:41.597171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc8b22-1a74-44f5-9aa2-3bdceb7704ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:41.598073 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc8b22-1a74-44f5-9aa2-3bdceb7704ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:41.598389 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc8b22-1a74-44f5-9aa2-3bdceb7704ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:41.598668 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc8b22-1a74-44f5-9aa2-3bdceb7704ce admin admin] PATH_INFO: `/v3/users/9f93a51f82a94e269bca847a4f629bf6` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:41.599833 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc8b22-1a74-44f5-9aa2-3bdceb7704ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f93a51f82a94e269bca847a4f629bf6)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:41.601783 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc8b22-1a74-44f5-9aa2-3bdceb7704ce admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:41.658143 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f5dc8b22-1a74-44f5-9aa2-3bdceb7704ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f93a51f82a94e269bca847a4f629bf6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:41.672002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f5dc8b22-1a74-44f5-9aa2-3bdceb7704ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f93a51f82a94e269bca847a4f629bf6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:41.672709 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f5dc8b22-1a74-44f5-9aa2-3bdceb7704ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f93a51f82a94e269bca847a4f629bf6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:41.680442 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f5dc8b22-1a74-44f5-9aa2-3bdceb7704ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f93a51f82a94e269bca847a4f629bf6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:41.693896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f5dc8b22-1a74-44f5-9aa2-3bdceb7704ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f93a51f82a94e269bca847a4f629bf6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:41.700289 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1023/2073] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:32:41 2022] DELETE /identity/v3/users/9f93a51f82a94e269bca847a4f629bf6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:41.704946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98dcd0-a337-4011-a84a-e4df91c43ce0 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:41.705399 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98dcd0-a337-4011-a84a-e4df91c43ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:41.705490 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98dcd0-a337-4011-a84a-e4df91c43ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:41.725079 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5c98dcd0-a337-4011-a84a-e4df91c43ce0 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:41.726310 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5c98dcd0-a337-4011-a84a-e4df91c43ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:41.783464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1051/2074] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:41.793634 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc8b22-1a74-44f5-9aa2-3bdceb7704ce admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:41.795144 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f5dc8b22-1a74-44f5-9aa2-3bdceb7704ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:41.851782 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc8b22-1a74-44f5-9aa2-3bdceb7704ce admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:41.855526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d0bb9c-94b7-44a2-b9ef-45f3c79e48b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:41.856137 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0bb9c-94b7-44a2-b9ef-45f3c79e48b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:41.856419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0bb9c-94b7-44a2-b9ef-45f3c79e48b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:41.856694 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0bb9c-94b7-44a2-b9ef-45f3c79e48b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:41.865737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d0bb9c-94b7-44a2-b9ef-45f3c79e48b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:41.867193 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d0bb9c-94b7-44a2-b9ef-45f3c79e48b8 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:41.876712 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1024/2075] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:42.231943 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98dcd0-a337-4011-a84a-e4df91c43ce0 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:42.234171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5c98dcd0-a337-4011-a84a-e4df91c43ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:42.248277 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98dcd0-a337-4011-a84a-e4df91c43ce0 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:42.252918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b5d2b-6492-4613-9963-c924aa3b38a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:42.253573 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b5d2b-6492-4613-9963-c924aa3b38a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:42.253833 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b5d2b-6492-4613-9963-c924aa3b38a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:42.254109 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b5d2b-6492-4613-9963-c924aa3b38a9 admin admin] PATH_INFO: `/v3/projects/fd8e0ed266414e2c8c2f177746b579da` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:42.255284 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b5d2b-6492-4613-9963-c924aa3b38a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd8e0ed266414e2c8c2f177746b579da)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:42.257191 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b5d2b-6492-4613-9963-c924aa3b38a9 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:42.306975 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-141b5d2b-6492-4613-9963-c924aa3b38a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd8e0ed266414e2c8c2f177746b579da', 'request_id': 'req-141b5d2b-6492-4613-9963-c924aa3b38a9'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:42.313714 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-141b5d2b-6492-4613-9963-c924aa3b38a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd8e0ed266414e2c8c2f177746b579da', 'request_id': 'req-141b5d2b-6492-4613-9963-c924aa3b38a9'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:42.319286 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-141b5d2b-6492-4613-9963-c924aa3b38a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd8e0ed266414e2c8c2f177746b579da', 'request_id': 'req-141b5d2b-6492-4613-9963-c924aa3b38a9'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:42.332934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-141b5d2b-6492-4613-9963-c924aa3b38a9 admin admin] The token cache is being invalidate because project fd8e0ed266414e2c8c2f177746b579da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:32:42.333222 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-141b5d2b-6492-4613-9963-c924aa3b38a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:42.333849 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-141b5d2b-6492-4613-9963-c924aa3b38a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:42.335356 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1052/2076] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:32:42 2022] DELETE /identity/v3/projects/fd8e0ed266414e2c8c2f177746b579da => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:32:42.340202 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eba97c-507e-4352-b46f-b12fb45ecfa2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:42.340495 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eba97c-507e-4352-b46f-b12fb45ecfa2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:42.340804 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eba97c-507e-4352-b46f-b12fb45ecfa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:42.360406 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-63eba97c-507e-4352-b46f-b12fb45ecfa2 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:42.361600 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-63eba97c-507e-4352-b46f-b12fb45ecfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:42.378457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:42.422596 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1025/2077] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:32:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:42.430108 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58c2bc-3ac1-41cb-9aec-952032614262 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:42.430424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58c2bc-3ac1-41cb-9aec-952032614262 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:42.430703 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58c2bc-3ac1-41cb-9aec-952032614262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:42.450460 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-8e58c2bc-3ac1-41cb-9aec-952032614262 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:42.451649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8e58c2bc-3ac1-41cb-9aec-952032614262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:42.459383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1053/2078] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:32:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:42.463457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eba97c-507e-4352-b46f-b12fb45ecfa2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:42.473250 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eba97c-507e-4352-b46f-b12fb45ecfa2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:42.477772 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eec16e-1238-40cc-ab12-5df570dfdfdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:42.478418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eec16e-1238-40cc-ab12-5df570dfdfdf admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:42.478685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eec16e-1238-40cc-ab12-5df570dfdfdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:42.478960 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eec16e-1238-40cc-ab12-5df570dfdfdf admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:42.480175 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eec16e-1238-40cc-ab12-5df570dfdfdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:32:42.480479 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eec16e-1238-40cc-ab12-5df570dfdfdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:42.482077 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eec16e-1238-40cc-ab12-5df570dfdfdf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:42.493437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1026/2079] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:32:42.498060 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58c2bc-3ac1-41cb-9aec-952032614262 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:42.507743 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58c2bc-3ac1-41cb-9aec-952032614262 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:42.512412 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1fef0-9d2e-49ba-b222-39c901c85045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:42.513007 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1fef0-9d2e-49ba-b222-39c901c85045 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:42.513289 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1fef0-9d2e-49ba-b222-39c901c85045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:42.513557 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1fef0-9d2e-49ba-b222-39c901c85045 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:42.514219 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f1fef0-9d2e-49ba-b222-39c901c85045 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:42.516107 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f1fef0-9d2e-49ba-b222-39c901c85045 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:42.543268 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1054/2080] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:32:42 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:42.547449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eec16e-1238-40cc-ab12-5df570dfdfdf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:42.557162 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eec16e-1238-40cc-ab12-5df570dfdfdf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:42.613338 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da323f5-312f-4c29-9484-01061344e478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:42.613942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da323f5-312f-4c29-9484-01061344e478 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:42.614253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da323f5-312f-4c29-9484-01061344e478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:42.614531 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da323f5-312f-4c29-9484-01061344e478 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:42.615151 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da323f5-312f-4c29-9484-01061344e478 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:42.617061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da323f5-312f-4c29-9484-01061344e478 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:42.638142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1027/2081] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:32:42 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:42.642419 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1fef0-9d2e-49ba-b222-39c901c85045 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:42.652682 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1fef0-9d2e-49ba-b222-39c901c85045 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:42.657153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5792b5b5-47d9-4b56-a1d9-cda88ff03718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:42.657732 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792b5b5-47d9-4b56-a1d9-cda88ff03718 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:42.658007 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792b5b5-47d9-4b56-a1d9-cda88ff03718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:42.658308 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792b5b5-47d9-4b56-a1d9-cda88ff03718 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:42.658950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5792b5b5-47d9-4b56-a1d9-cda88ff03718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:42.660773 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5792b5b5-47d9-4b56-a1d9-cda88ff03718 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:42.672200 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1055/2082] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:42.676080 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da323f5-312f-4c29-9484-01061344e478 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:42.687295 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da323f5-312f-4c29-9484-01061344e478 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:42.692222 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc476145-0893-4b3d-be0a-ddae661f6bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:42.692823 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc476145-0893-4b3d-be0a-ddae661f6bcd admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:42.693104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc476145-0893-4b3d-be0a-ddae661f6bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:42.693374 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc476145-0893-4b3d-be0a-ddae661f6bcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:42.694131 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc476145-0893-4b3d-be0a-ddae661f6bcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:42.695699 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc476145-0893-4b3d-be0a-ddae661f6bcd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:42.706350 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1028/2083] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:42.710587 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5792b5b5-47d9-4b56-a1d9-cda88ff03718 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:42.720977 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5792b5b5-47d9-4b56-a1d9-cda88ff03718 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:42.725578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20e0a3-1186-4c97-832f-94be1c10dd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:42.726165 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20e0a3-1186-4c97-832f-94be1c10dd6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:42.726473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20e0a3-1186-4c97-832f-94be1c10dd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:42.726748 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20e0a3-1186-4c97-832f-94be1c10dd6e admin admin] PATH_INFO: `/v3/projects/35d5da7bdb714b29ac21374c7e886553/users/3bb0c902b8fa4d98ad0c50ac324ef8ca/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:42.742599 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea20e0a3-1186-4c97-832f-94be1c10dd6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=35d5da7bdb714b29ac21374c7e886553, user_id=3bb0c902b8fa4d98ad0c50ac324ef8ca, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:42.745295 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea20e0a3-1186-4c97-832f-94be1c10dd6e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:42.759333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1056/2084] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:32:42 2022] PUT /identity/v3/projects/35d5da7bdb714b29ac21374c7e886553/users/3bb0c902b8fa4d98ad0c50ac324ef8ca/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:32:42.769722 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4759a48-adb6-423a-b961-2fdbaedd3a78 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:42.770012 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4759a48-adb6-423a-b961-2fdbaedd3a78 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:42.770326 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4759a48-adb6-423a-b961-2fdbaedd3a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:42.801093 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a4759a48-adb6-423a-b961-2fdbaedd3a78 None None] MFA Rules not processed for user `3bb0c902b8fa4d98ad0c50ac324ef8ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:42.802369 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a4759a48-adb6-423a-b961-2fdbaedd3a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:42.866628 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1029/2085] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:32:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:42.877444 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20e0a3-1186-4c97-832f-94be1c10dd6e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:42.879204 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ea20e0a3-1186-4c97-832f-94be1c10dd6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:42.901639 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:42.937836 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20e0a3-1186-4c97-832f-94be1c10dd6e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:42.941422 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce400f4a-f450-4a54-a48d-45345b36e551 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:42.942020 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce400f4a-f450-4a54-a48d-45345b36e551 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:42.942336 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce400f4a-f450-4a54-a48d-45345b36e551 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:42.942608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce400f4a-f450-4a54-a48d-45345b36e551 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:42.950833 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce400f4a-f450-4a54-a48d-45345b36e551 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:42.952240 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce400f4a-f450-4a54-a48d-45345b36e551 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:42.960721 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1057/2086] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:42.985480 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4759a48-adb6-423a-b961-2fdbaedd3a78 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:43.003945 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:43.048958 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4759a48-adb6-423a-b961-2fdbaedd3a78 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:43.053738 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042021c6-c6d2-4352-a25a-69e755b87413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:43.054404 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042021c6-c6d2-4352-a25a-69e755b87413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:43.054690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042021c6-c6d2-4352-a25a-69e755b87413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:43.054966 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042021c6-c6d2-4352-a25a-69e755b87413 admin admin] PATH_INFO: `/v3/users/3bb0c902b8fa4d98ad0c50ac324ef8ca` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:43.056163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042021c6-c6d2-4352-a25a-69e755b87413 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bb0c902b8fa4d98ad0c50ac324ef8ca)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:43.058169 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042021c6-c6d2-4352-a25a-69e755b87413 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:43.105067 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-042021c6-c6d2-4352-a25a-69e755b87413 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bb0c902b8fa4d98ad0c50ac324ef8ca'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:43.112066 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-042021c6-c6d2-4352-a25a-69e755b87413 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bb0c902b8fa4d98ad0c50ac324ef8ca'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:43.112760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-042021c6-c6d2-4352-a25a-69e755b87413 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bb0c902b8fa4d98ad0c50ac324ef8ca'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:43.120302 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-042021c6-c6d2-4352-a25a-69e755b87413 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bb0c902b8fa4d98ad0c50ac324ef8ca'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:43.131983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-042021c6-c6d2-4352-a25a-69e755b87413 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bb0c902b8fa4d98ad0c50ac324ef8ca'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:43.138646 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1030/2087] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:32:42 2022] DELETE /identity/v3/users/3bb0c902b8fa4d98ad0c50ac324ef8ca => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:43.143061 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b250161-3500-450f-8f9f-7b05d81e7c49 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:43.143340 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b250161-3500-450f-8f9f-7b05d81e7c49 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:43.143610 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b250161-3500-450f-8f9f-7b05d81e7c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:43.163369 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-0b250161-3500-450f-8f9f-7b05d81e7c49 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:43.164551 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0b250161-3500-450f-8f9f-7b05d81e7c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:43.221592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1058/2088] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:43.231828 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042021c6-c6d2-4352-a25a-69e755b87413 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:43.233153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-042021c6-c6d2-4352-a25a-69e755b87413 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:43.290358 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042021c6-c6d2-4352-a25a-69e755b87413 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:43.294981 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49da28-7c9d-433c-9566-2e6d565a72ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:43.295759 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49da28-7c9d-433c-9566-2e6d565a72ec service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:43.296041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49da28-7c9d-433c-9566-2e6d565a72ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:43.296410 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49da28-7c9d-433c-9566-2e6d565a72ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:43.305950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b49da28-7c9d-433c-9566-2e6d565a72ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:43.307964 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b49da28-7c9d-433c-9566-2e6d565a72ec service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:43.317745 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1031/2089] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:43.623401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b250161-3500-450f-8f9f-7b05d81e7c49 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:43.624826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0b250161-3500-450f-8f9f-7b05d81e7c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:43.636400 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b250161-3500-450f-8f9f-7b05d81e7c49 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:43.640984 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b14065a-b1fd-4f6e-89b8-9cd96703568f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:43.641604 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14065a-b1fd-4f6e-89b8-9cd96703568f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:43.642561 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14065a-b1fd-4f6e-89b8-9cd96703568f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:43.642851 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14065a-b1fd-4f6e-89b8-9cd96703568f admin admin] PATH_INFO: `/v3/projects/35d5da7bdb714b29ac21374c7e886553` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:43.644158 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b14065a-b1fd-4f6e-89b8-9cd96703568f admin admin] RBAC: Authorizing `identity:delete_project(project_id=35d5da7bdb714b29ac21374c7e886553)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:43.646059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b14065a-b1fd-4f6e-89b8-9cd96703568f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:43.695150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0b14065a-b1fd-4f6e-89b8-9cd96703568f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35d5da7bdb714b29ac21374c7e886553', 'request_id': 'req-0b14065a-b1fd-4f6e-89b8-9cd96703568f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:43.701362 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0b14065a-b1fd-4f6e-89b8-9cd96703568f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35d5da7bdb714b29ac21374c7e886553', 'request_id': 'req-0b14065a-b1fd-4f6e-89b8-9cd96703568f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:43.706880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0b14065a-b1fd-4f6e-89b8-9cd96703568f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35d5da7bdb714b29ac21374c7e886553', 'request_id': 'req-0b14065a-b1fd-4f6e-89b8-9cd96703568f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:43.721020 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0b14065a-b1fd-4f6e-89b8-9cd96703568f admin admin] The token cache is being invalidate because project 35d5da7bdb714b29ac21374c7e886553 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:32:43.721632 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0b14065a-b1fd-4f6e-89b8-9cd96703568f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:43.722092 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0b14065a-b1fd-4f6e-89b8-9cd96703568f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:43.723680 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1059/2090] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:32:43 2022] DELETE /identity/v3/projects/35d5da7bdb714b29ac21374c7e886553 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:43.728548 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb5dd5-adea-41de-949f-75bd494d632e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:43.728947 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb5dd5-adea-41de-949f-75bd494d632e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:43.729090 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb5dd5-adea-41de-949f-75bd494d632e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:43.749136 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-04eb5dd5-adea-41de-949f-75bd494d632e None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:43.750278 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-04eb5dd5-adea-41de-949f-75bd494d632e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:43.782811 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:43.809729 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1032/2091] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:43.818472 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a185ba-4b29-49d1-953f-0098916c6ee7 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:43.818758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a185ba-4b29-49d1-953f-0098916c6ee7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:43.819030 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a185ba-4b29-49d1-953f-0098916c6ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:43.839011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-08a185ba-4b29-49d1-953f-0098916c6ee7 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:43.840200 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-08a185ba-4b29-49d1-953f-0098916c6ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:43.848024 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1060/2092] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:43.852275 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb5dd5-adea-41de-949f-75bd494d632e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:43.863128 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb5dd5-adea-41de-949f-75bd494d632e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:43.867750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62780928-da83-475a-b95c-57da84396029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:43.868383 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62780928-da83-475a-b95c-57da84396029 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:43.868667 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62780928-da83-475a-b95c-57da84396029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:43.868944 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62780928-da83-475a-b95c-57da84396029 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:43.869611 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62780928-da83-475a-b95c-57da84396029 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:32:43.869921 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62780928-da83-475a-b95c-57da84396029 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:43.871579 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62780928-da83-475a-b95c-57da84396029 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:43.884038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1033/2093] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:32:43.888928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a185ba-4b29-49d1-953f-0098916c6ee7 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:43.898387 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a185ba-4b29-49d1-953f-0098916c6ee7 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:43.903830 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af8ab5-ef60-4d4a-89b7-1dd5c25346da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:43.904421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af8ab5-ef60-4d4a-89b7-1dd5c25346da admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:43.904694 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af8ab5-ef60-4d4a-89b7-1dd5c25346da admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:43.904961 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af8ab5-ef60-4d4a-89b7-1dd5c25346da admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:43.905599 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af8ab5-ef60-4d4a-89b7-1dd5c25346da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:43.907635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af8ab5-ef60-4d4a-89b7-1dd5c25346da admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:43.935625 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1061/2094] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:32:43 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:43.939921 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62780928-da83-475a-b95c-57da84396029 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:43.951262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62780928-da83-475a-b95c-57da84396029 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:44.009867 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0f644-a01e-4fec-82e5-516aeed92b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:44.011031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0f644-a01e-4fec-82e5-516aeed92b1a admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:44.011314 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0f644-a01e-4fec-82e5-516aeed92b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:44.011588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0f644-a01e-4fec-82e5-516aeed92b1a admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:44.012224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe0f644-a01e-4fec-82e5-516aeed92b1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:44.014134 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe0f644-a01e-4fec-82e5-516aeed92b1a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:44.040203 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1034/2095] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:32:43 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:44.045373 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af8ab5-ef60-4d4a-89b7-1dd5c25346da admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:44.053376 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af8ab5-ef60-4d4a-89b7-1dd5c25346da admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:44.058239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0835f8-f501-4d28-a0f9-2978bae09608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:44.058845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0835f8-f501-4d28-a0f9-2978bae09608 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:44.059125 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0835f8-f501-4d28-a0f9-2978bae09608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:44.059398 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0835f8-f501-4d28-a0f9-2978bae09608 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:44.060492 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0835f8-f501-4d28-a0f9-2978bae09608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:44.062299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0835f8-f501-4d28-a0f9-2978bae09608 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:44.074459 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1062/2096] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:44.078544 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0f644-a01e-4fec-82e5-516aeed92b1a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:44.089281 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0f644-a01e-4fec-82e5-516aeed92b1a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:44.093813 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c3f20-3137-4432-a101-b889ccebd8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:44.094433 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c3f20-3137-4432-a101-b889ccebd8bb admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:44.094707 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c3f20-3137-4432-a101-b889ccebd8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:44.094976 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c3f20-3137-4432-a101-b889ccebd8bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:44.095616 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c3f20-3137-4432-a101-b889ccebd8bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:44.097193 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c3f20-3137-4432-a101-b889ccebd8bb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:44.107630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1035/2097] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:44.111712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0835f8-f501-4d28-a0f9-2978bae09608 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:44.121643 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0835f8-f501-4d28-a0f9-2978bae09608 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:44.126234 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeee6a9-35be-4470-a83c-6e211034c953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:44.126879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeee6a9-35be-4470-a83c-6e211034c953 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:44.127164 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeee6a9-35be-4470-a83c-6e211034c953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:44.127442 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeee6a9-35be-4470-a83c-6e211034c953 admin admin] PATH_INFO: `/v3/projects/73bf944cd36148b7bb4a5516badc3a26/users/916c538de1004cbcb9c21aa88fa9bed0/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:44.129633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:44.144509 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeee6a9-35be-4470-a83c-6e211034c953 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73bf944cd36148b7bb4a5516badc3a26, user_id=916c538de1004cbcb9c21aa88fa9bed0, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:44.147215 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeee6a9-35be-4470-a83c-6e211034c953 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:44.160239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1063/2098] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:32:44 2022] PUT /identity/v3/projects/73bf944cd36148b7bb4a5516badc3a26/users/916c538de1004cbcb9c21aa88fa9bed0/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:44.170436 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fdc1c-3d33-4a52-85e6-9bb11cf61918 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:44.170729 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fdc1c-3d33-4a52-85e6-9bb11cf61918 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:44.171005 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fdc1c-3d33-4a52-85e6-9bb11cf61918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:44.200348 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0c1fdc1c-3d33-4a52-85e6-9bb11cf61918 None None] MFA Rules not processed for user `916c538de1004cbcb9c21aa88fa9bed0`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:44.201582 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0c1fdc1c-3d33-4a52-85e6-9bb11cf61918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:44.265968 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1036/2099] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:32:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:44.276735 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeee6a9-35be-4470-a83c-6e211034c953 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:44.278451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ffeee6a9-35be-4470-a83c-6e211034c953 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:44.337577 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeee6a9-35be-4470-a83c-6e211034c953 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:44.341230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d53ac-ab77-4ee2-a321-03fbdf90af3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:44.341829 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d53ac-ab77-4ee2-a321-03fbdf90af3d service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:44.342108 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d53ac-ab77-4ee2-a321-03fbdf90af3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:44.342414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d53ac-ab77-4ee2-a321-03fbdf90af3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:44.350783 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d53ac-ab77-4ee2-a321-03fbdf90af3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:44.352239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d53ac-ab77-4ee2-a321-03fbdf90af3d service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:44.361869 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1064/2100] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:53.417023 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fdc1c-3d33-4a52-85e6-9bb11cf61918 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:53.418498 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0c1fdc1c-3d33-4a52-85e6-9bb11cf61918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:53.458099 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:53.485190 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fdc1c-3d33-4a52-85e6-9bb11cf61918 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:53.491508 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161cba9b-ecc3-4475-8e3e-ac8a3a441d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:53.492085 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cba9b-ecc3-4475-8e3e-ac8a3a441d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:53.492544 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cba9b-ecc3-4475-8e3e-ac8a3a441d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:53.492711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cba9b-ecc3-4475-8e3e-ac8a3a441d82 admin admin] PATH_INFO: `/v3/users/c3b955c057614d3fbcc01ecfecdb4931` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:53.493990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161cba9b-ecc3-4475-8e3e-ac8a3a441d82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3b955c057614d3fbcc01ecfecdb4931)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:53.496718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161cba9b-ecc3-4475-8e3e-ac8a3a441d82 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:53.546733 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-161cba9b-ecc3-4475-8e3e-ac8a3a441d82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3b955c057614d3fbcc01ecfecdb4931'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:53.555384 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-161cba9b-ecc3-4475-8e3e-ac8a3a441d82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3b955c057614d3fbcc01ecfecdb4931'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:53.555995 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-161cba9b-ecc3-4475-8e3e-ac8a3a441d82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3b955c057614d3fbcc01ecfecdb4931'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:53.565017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-161cba9b-ecc3-4475-8e3e-ac8a3a441d82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3b955c057614d3fbcc01ecfecdb4931'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:53.578479 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-161cba9b-ecc3-4475-8e3e-ac8a3a441d82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3b955c057614d3fbcc01ecfecdb4931'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:53.584942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1037/2101] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:32:53 2022] DELETE /identity/v3/users/c3b955c057614d3fbcc01ecfecdb4931 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:53.589575 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49861c7f-152b-4b66-adbb-ace7ea0ae32e None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:53.590081 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49861c7f-152b-4b66-adbb-ace7ea0ae32e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:53.590220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49861c7f-152b-4b66-adbb-ace7ea0ae32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:53.612179 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-49861c7f-152b-4b66-adbb-ace7ea0ae32e None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:53.613325 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-49861c7f-152b-4b66-adbb-ace7ea0ae32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:53.680429 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1065/2102] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:53.690843 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161cba9b-ecc3-4475-8e3e-ac8a3a441d82 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:53.692251 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-161cba9b-ecc3-4475-8e3e-ac8a3a441d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:53.750257 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161cba9b-ecc3-4475-8e3e-ac8a3a441d82 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:53.753910 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe76067-f576-4a79-b6e6-569d019ec054 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:53.754572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe76067-f576-4a79-b6e6-569d019ec054 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:53.754862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe76067-f576-4a79-b6e6-569d019ec054 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:53.755164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe76067-f576-4a79-b6e6-569d019ec054 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:53.764089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe76067-f576-4a79-b6e6-569d019ec054 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:53.765548 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe76067-f576-4a79-b6e6-569d019ec054 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:53.774864 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1038/2103] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:54.083063 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49861c7f-152b-4b66-adbb-ace7ea0ae32e None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:54.084499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-49861c7f-152b-4b66-adbb-ace7ea0ae32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:54.098126 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49861c7f-152b-4b66-adbb-ace7ea0ae32e None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:54.102742 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae6707-ce86-411b-8dca-60fa615b7020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:54.103439 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae6707-ce86-411b-8dca-60fa615b7020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:54.103779 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae6707-ce86-411b-8dca-60fa615b7020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:54.104114 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae6707-ce86-411b-8dca-60fa615b7020 admin admin] PATH_INFO: `/v3/projects/59256b7f61424ae4b6038009232caac8` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:54.105987 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae6707-ce86-411b-8dca-60fa615b7020 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59256b7f61424ae4b6038009232caac8)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:54.108265 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae6707-ce86-411b-8dca-60fa615b7020 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:54.154601 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-beae6707-ce86-411b-8dca-60fa615b7020 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59256b7f61424ae4b6038009232caac8', 'request_id': 'req-beae6707-ce86-411b-8dca-60fa615b7020'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:54.160959 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-beae6707-ce86-411b-8dca-60fa615b7020 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59256b7f61424ae4b6038009232caac8', 'request_id': 'req-beae6707-ce86-411b-8dca-60fa615b7020'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:54.166550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-beae6707-ce86-411b-8dca-60fa615b7020 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59256b7f61424ae4b6038009232caac8', 'request_id': 'req-beae6707-ce86-411b-8dca-60fa615b7020'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:54.179323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-beae6707-ce86-411b-8dca-60fa615b7020 admin admin] The token cache is being invalidate because project 59256b7f61424ae4b6038009232caac8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:32:54.179615 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-beae6707-ce86-411b-8dca-60fa615b7020 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:54.180241 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-beae6707-ce86-411b-8dca-60fa615b7020 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:54.181719 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1066/2104] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:32:54 2022] DELETE /identity/v3/projects/59256b7f61424ae4b6038009232caac8 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:00.705944 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe76067-f576-4a79-b6e6-569d019ec054 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:00.707727 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cfe76067-f576-4a79-b6e6-569d019ec054 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:00.722038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe76067-f576-4a79-b6e6-569d019ec054 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:00.795494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:00.799824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd9b8c-f93e-4b51-9805-7d4caa9bfa28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:00.800655 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd9b8c-f93e-4b51-9805-7d4caa9bfa28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:00.800943 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd9b8c-f93e-4b51-9805-7d4caa9bfa28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:00.801223 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd9b8c-f93e-4b51-9805-7d4caa9bfa28 admin admin] PATH_INFO: `/v3/users/916c538de1004cbcb9c21aa88fa9bed0` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:00.803240 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdd9b8c-f93e-4b51-9805-7d4caa9bfa28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=916c538de1004cbcb9c21aa88fa9bed0)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:00.805570 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdd9b8c-f93e-4b51-9805-7d4caa9bfa28 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:00.873483 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0bdd9b8c-f93e-4b51-9805-7d4caa9bfa28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '916c538de1004cbcb9c21aa88fa9bed0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:00.881561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0bdd9b8c-f93e-4b51-9805-7d4caa9bfa28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '916c538de1004cbcb9c21aa88fa9bed0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:00.882479 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0bdd9b8c-f93e-4b51-9805-7d4caa9bfa28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '916c538de1004cbcb9c21aa88fa9bed0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:00.892772 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0bdd9b8c-f93e-4b51-9805-7d4caa9bfa28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '916c538de1004cbcb9c21aa88fa9bed0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:00.908362 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0bdd9b8c-f93e-4b51-9805-7d4caa9bfa28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '916c538de1004cbcb9c21aa88fa9bed0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:00.915871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1039/2105] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:33:00 2022] DELETE /identity/v3/users/916c538de1004cbcb9c21aa88fa9bed0 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:00.920559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61983df-e954-4041-b2d3-7efefd3c68a6 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:00.921033 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61983df-e954-4041-b2d3-7efefd3c68a6 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:00.921432 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61983df-e954-4041-b2d3-7efefd3c68a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:00.946150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f61983df-e954-4041-b2d3-7efefd3c68a6 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:00.947499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f61983df-e954-4041-b2d3-7efefd3c68a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:00.953368 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:01.037076 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1067/2106] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:33:01.047099 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd9b8c-f93e-4b51-9805-7d4caa9bfa28 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:01.048823 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0bdd9b8c-f93e-4b51-9805-7d4caa9bfa28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:01.112874 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd9b8c-f93e-4b51-9805-7d4caa9bfa28 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:01.116743 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdaba3c-4de4-429d-8280-44e83eadf5d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:01.117344 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdaba3c-4de4-429d-8280-44e83eadf5d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:01.117618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdaba3c-4de4-429d-8280-44e83eadf5d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:01.117889 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdaba3c-4de4-429d-8280-44e83eadf5d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:01.127725 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdaba3c-4de4-429d-8280-44e83eadf5d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:01.129147 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdaba3c-4de4-429d-8280-44e83eadf5d4 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:01.138534 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1040/2107] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:01.218418 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1886c2-51c9-4eb2-b64b-1ebcac5aa5ad None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:01.218706 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1886c2-51c9-4eb2-b64b-1ebcac5aa5ad None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:01.218982 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1886c2-51c9-4eb2-b64b-1ebcac5aa5ad None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:01.220719 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1068/2108] 10.222.0.58 () {54 vars in 921 bytes} [Thu Sep 1 16:33:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:33:01.251581 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de75f1-0765-41eb-9daf-136bce7e7377 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:01.251882 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de75f1-0765-41eb-9daf-136bce7e7377 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:01.252686 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de75f1-0765-41eb-9daf-136bce7e7377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:01.272792 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-85de75f1-0765-41eb-9daf-136bce7e7377 None None] MFA Rules not processed for user `e12302cdb2854b0684179647a6dadf6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:01.274034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-85de75f1-0765-41eb-9daf-136bce7e7377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:01.295298 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1886c2-51c9-4eb2-b64b-1ebcac5aa5ad None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:01.297983 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3c1886c2-51c9-4eb2-b64b-1ebcac5aa5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:01.316894 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1886c2-51c9-4eb2-b64b-1ebcac5aa5ad None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:01.323683 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f4c37-7520-4d31-b49d-b08275bfd230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:01.324446 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f4c37-7520-4d31-b49d-b08275bfd230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:01.324986 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f4c37-7520-4d31-b49d-b08275bfd230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:01.325365 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f4c37-7520-4d31-b49d-b08275bfd230 admin admin] PATH_INFO: `/v3/users/f69ac454b5a64e6d83188ad2b8e107d4` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:01.326688 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f4c37-7520-4d31-b49d-b08275bfd230 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f69ac454b5a64e6d83188ad2b8e107d4)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:01.330754 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f4c37-7520-4d31-b49d-b08275bfd230 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:01.381638 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1041/2109] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Sep 1 16:33:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:01.395459 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-718f4c37-7520-4d31-b49d-b08275bfd230 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f69ac454b5a64e6d83188ad2b8e107d4'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:01.403413 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de75f1-0765-41eb-9daf-136bce7e7377 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:01.405021 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-85de75f1-0765-41eb-9daf-136bce7e7377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:01.412945 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-718f4c37-7520-4d31-b49d-b08275bfd230 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f69ac454b5a64e6d83188ad2b8e107d4'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:01.415260 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-718f4c37-7520-4d31-b49d-b08275bfd230 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f69ac454b5a64e6d83188ad2b8e107d4'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:01.416097 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-718f4c37-7520-4d31-b49d-b08275bfd230 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f69ac454b5a64e6d83188ad2b8e107d4'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:01.425842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de75f1-0765-41eb-9daf-136bce7e7377 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:01.433507 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ec132-e253-4cb5-a17b-72bb9e93c279 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12302cdb2854b0684179647a6dadf6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:01.434321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ec132-e253-4cb5-a17b-72bb9e93c279 service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:01.434610 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ec132-e253-4cb5-a17b-72bb9e93c279 service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:01.434913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ec132-e253-4cb5-a17b-72bb9e93c279 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:01.436894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a12ec132-e253-4cb5-a17b-72bb9e93c279 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:01.441011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-718f4c37-7520-4d31-b49d-b08275bfd230 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f69ac454b5a64e6d83188ad2b8e107d4'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:01.448761 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ec132-e253-4cb5-a17b-72bb9e93c279 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:01.450445 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ec132-e253-4cb5-a17b-72bb9e93c279 service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:01.460892 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1042/2111] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:33:01.462573 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1069/2111] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:33:01 2022] DELETE /identity/v3/users/f69ac454b5a64e6d83188ad2b8e107d4 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:01.506299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ec132-e253-4cb5-a17b-72bb9e93c279 service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:01.508389 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a12ec132-e253-4cb5-a17b-72bb9e93c279 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:01.520155 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:01.530022 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ec132-e253-4cb5-a17b-72bb9e93c279 service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:01.602956 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ba59a-5513-4e74-a1c5-4ce049bd1713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:01.603603 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ba59a-5513-4e74-a1c5-4ce049bd1713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:01.604520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ba59a-5513-4e74-a1c5-4ce049bd1713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:01.604798 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ba59a-5513-4e74-a1c5-4ce049bd1713 admin admin] PATH_INFO: `/v3/projects/73bf944cd36148b7bb4a5516badc3a26` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:01.605926 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ba59a-5513-4e74-a1c5-4ce049bd1713 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73bf944cd36148b7bb4a5516badc3a26)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:01.607898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ba59a-5513-4e74-a1c5-4ce049bd1713 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:01.657925 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f4c37-7520-4d31-b49d-b08275bfd230 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:01.659931 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4b2ba59a-5513-4e74-a1c5-4ce049bd1713 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73bf944cd36148b7bb4a5516badc3a26', 'request_id': 'req-4b2ba59a-5513-4e74-a1c5-4ce049bd1713'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:01.660268 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-718f4c37-7520-4d31-b49d-b08275bfd230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:01.667864 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4b2ba59a-5513-4e74-a1c5-4ce049bd1713 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73bf944cd36148b7bb4a5516badc3a26', 'request_id': 'req-4b2ba59a-5513-4e74-a1c5-4ce049bd1713'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:01.677574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f4c37-7520-4d31-b49d-b08275bfd230 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:01.677776 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4b2ba59a-5513-4e74-a1c5-4ce049bd1713 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73bf944cd36148b7bb4a5516badc3a26', 'request_id': 'req-4b2ba59a-5513-4e74-a1c5-4ce049bd1713'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:01.691023 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4b2ba59a-5513-4e74-a1c5-4ce049bd1713 admin admin] The token cache is being invalidate because project 73bf944cd36148b7bb4a5516badc3a26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:33:01.691727 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4b2ba59a-5513-4e74-a1c5-4ce049bd1713 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:01.693084 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4b2ba59a-5513-4e74-a1c5-4ce049bd1713 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:01.694727 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1043/2112] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:33:01 2022] DELETE /identity/v3/projects/73bf944cd36148b7bb4a5516badc3a26 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:01.745072 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358403c9-e767-4731-a442-92b4e6a87f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:01.745884 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358403c9-e767-4731-a442-92b4e6a87f02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:01.746169 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358403c9-e767-4731-a442-92b4e6a87f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:01.746511 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358403c9-e767-4731-a442-92b4e6a87f02 admin admin] PATH_INFO: `/v3/projects/19bb3b3c67024a7bbe30ba1b59e7f50a` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:01.747771 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358403c9-e767-4731-a442-92b4e6a87f02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19bb3b3c67024a7bbe30ba1b59e7f50a)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:01.749796 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358403c9-e767-4731-a442-92b4e6a87f02 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:01.777745 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:01.805967 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-358403c9-e767-4731-a442-92b4e6a87f02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19bb3b3c67024a7bbe30ba1b59e7f50a', 'request_id': 'req-358403c9-e767-4731-a442-92b4e6a87f02'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:01.812912 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-358403c9-e767-4731-a442-92b4e6a87f02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19bb3b3c67024a7bbe30ba1b59e7f50a', 'request_id': 'req-358403c9-e767-4731-a442-92b4e6a87f02'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:01.819299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-358403c9-e767-4731-a442-92b4e6a87f02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19bb3b3c67024a7bbe30ba1b59e7f50a', 'request_id': 'req-358403c9-e767-4731-a442-92b4e6a87f02'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:01.835131 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-358403c9-e767-4731-a442-92b4e6a87f02 admin admin] The token cache is being invalidate because project 19bb3b3c67024a7bbe30ba1b59e7f50a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:33:01.835427 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-358403c9-e767-4731-a442-92b4e6a87f02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:01.836095 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-358403c9-e767-4731-a442-92b4e6a87f02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:01.837595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1070/2113] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:33:01 2022] DELETE /identity/v3/projects/19bb3b3c67024a7bbe30ba1b59e7f50a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:01.843257 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b349541-5998-4fe5-a23b-61fc0f63b41d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:01.843774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b349541-5998-4fe5-a23b-61fc0f63b41d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:01.843887 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b349541-5998-4fe5-a23b-61fc0f63b41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:01.866305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1b349541-5998-4fe5-a23b-61fc0f63b41d None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:01.867496 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1b349541-5998-4fe5-a23b-61fc0f63b41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:01.928635 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1044/2114] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:01.937638 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722236d5-1668-4707-acac-fc0c7437cc81 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:01.938136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722236d5-1668-4707-acac-fc0c7437cc81 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:01.938282 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722236d5-1668-4707-acac-fc0c7437cc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:01.958622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-722236d5-1668-4707-acac-fc0c7437cc81 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:01.959831 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-722236d5-1668-4707-acac-fc0c7437cc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:01.968654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1071/2115] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:01.972896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b349541-5998-4fe5-a23b-61fc0f63b41d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:01.982703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b349541-5998-4fe5-a23b-61fc0f63b41d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:01.987302 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93077eb-ceff-4f73-ad11-a1a03e2185f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:01.987894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93077eb-ceff-4f73-ad11-a1a03e2185f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:01.988178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93077eb-ceff-4f73-ad11-a1a03e2185f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:01.988455 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93077eb-ceff-4f73-ad11-a1a03e2185f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:01.989101 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93077eb-ceff-4f73-ad11-a1a03e2185f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:33:01.989408 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93077eb-ceff-4f73-ad11-a1a03e2185f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:01.991040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93077eb-ceff-4f73-ad11-a1a03e2185f7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:02.002294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1045/2116] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:33:02.006348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722236d5-1668-4707-acac-fc0c7437cc81 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:02.015951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722236d5-1668-4707-acac-fc0c7437cc81 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:02.020525 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f978b08-cf12-44bc-b535-0279058bad31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:02.021107 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f978b08-cf12-44bc-b535-0279058bad31 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:02.021384 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f978b08-cf12-44bc-b535-0279058bad31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:02.021656 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f978b08-cf12-44bc-b535-0279058bad31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:02.022311 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f978b08-cf12-44bc-b535-0279058bad31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:02.024180 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f978b08-cf12-44bc-b535-0279058bad31 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:02.057034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1072/2117] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:02 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:02.061406 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93077eb-ceff-4f73-ad11-a1a03e2185f7 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:02.071701 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93077eb-ceff-4f73-ad11-a1a03e2185f7 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:02.091890 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:02.136915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac261c4-02de-4881-a3bd-65669904681b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:02.137533 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac261c4-02de-4881-a3bd-65669904681b admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:02.137841 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac261c4-02de-4881-a3bd-65669904681b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:02.138114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac261c4-02de-4881-a3bd-65669904681b admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:02.138794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac261c4-02de-4881-a3bd-65669904681b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:02.140762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac261c4-02de-4881-a3bd-65669904681b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:02.163884 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1046/2118] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:02 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:02.168095 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f978b08-cf12-44bc-b535-0279058bad31 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:02.178721 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f978b08-cf12-44bc-b535-0279058bad31 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:02.183802 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6ff50-9b4c-4c7d-9c83-994574d99d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:02.184387 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6ff50-9b4c-4c7d-9c83-994574d99d82 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:02.184667 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6ff50-9b4c-4c7d-9c83-994574d99d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:02.184941 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6ff50-9b4c-4c7d-9c83-994574d99d82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:02.185599 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6ff50-9b4c-4c7d-9c83-994574d99d82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:02.187214 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6ff50-9b4c-4c7d-9c83-994574d99d82 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:02.197082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1073/2119] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:02.201587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac261c4-02de-4881-a3bd-65669904681b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:02.211735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac261c4-02de-4881-a3bd-65669904681b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:02.216315 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b01190-8777-4fb6-9088-6ffefbfab6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:02.216904 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b01190-8777-4fb6-9088-6ffefbfab6df admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:02.217185 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b01190-8777-4fb6-9088-6ffefbfab6df admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:02.217460 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b01190-8777-4fb6-9088-6ffefbfab6df admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:02.218100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b01190-8777-4fb6-9088-6ffefbfab6df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:02.219718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b01190-8777-4fb6-9088-6ffefbfab6df admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:02.230134 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1047/2120] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:02.234011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6ff50-9b4c-4c7d-9c83-994574d99d82 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:02.244851 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6ff50-9b4c-4c7d-9c83-994574d99d82 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:02.249356 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d0db86-2b78-4b21-975d-ec32cb167ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:02.249991 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0db86-2b78-4b21-975d-ec32cb167ecd admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:02.250311 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0db86-2b78-4b21-975d-ec32cb167ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:02.250589 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0db86-2b78-4b21-975d-ec32cb167ecd admin admin] PATH_INFO: `/v3/projects/3dcbfb9669934bbcaaceb6f43dd44f5e/users/d3aa5a36fc084259b48466cd893e036c/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:02.265367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d0db86-2b78-4b21-975d-ec32cb167ecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dcbfb9669934bbcaaceb6f43dd44f5e, user_id=d3aa5a36fc084259b48466cd893e036c, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:02.268047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d0db86-2b78-4b21-975d-ec32cb167ecd admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:02.280730 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1074/2121] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:02 2022] PUT /identity/v3/projects/3dcbfb9669934bbcaaceb6f43dd44f5e/users/d3aa5a36fc084259b48466cd893e036c/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:02.292204 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55080abf-a369-4ce7-aa6e-3d2da493cf4b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:02.292520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55080abf-a369-4ce7-aa6e-3d2da493cf4b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:02.292797 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55080abf-a369-4ce7-aa6e-3d2da493cf4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:02.320443 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-55080abf-a369-4ce7-aa6e-3d2da493cf4b None None] MFA Rules not processed for user `d3aa5a36fc084259b48466cd893e036c`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:02.321643 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-55080abf-a369-4ce7-aa6e-3d2da493cf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:02.384273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1048/2122] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:02.395844 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d0db86-2b78-4b21-975d-ec32cb167ecd admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:02.397510 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-89d0db86-2b78-4b21-975d-ec32cb167ecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:02.472054 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d0db86-2b78-4b21-975d-ec32cb167ecd admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:02.477535 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa73e28-2105-4550-a18b-21b50a2222d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:02.478130 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa73e28-2105-4550-a18b-21b50a2222d7 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:02.478436 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa73e28-2105-4550-a18b-21b50a2222d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:02.478710 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa73e28-2105-4550-a18b-21b50a2222d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:02.486775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa73e28-2105-4550-a18b-21b50a2222d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:02.488352 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa73e28-2105-4550-a18b-21b50a2222d7 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:02.497891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1075/2123] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:12.381526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=57,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:12.493303 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=76,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:12.890415 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55080abf-a369-4ce7-aa6e-3d2da493cf4b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:12.952225 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55080abf-a369-4ce7-aa6e-3d2da493cf4b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:12.956768 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acceaf70-5182-49a4-819b-38ab575fdab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:12.957390 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acceaf70-5182-49a4-819b-38ab575fdab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:12.957671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acceaf70-5182-49a4-819b-38ab575fdab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:12.958090 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acceaf70-5182-49a4-819b-38ab575fdab9 admin admin] PATH_INFO: `/v3/users/d3aa5a36fc084259b48466cd893e036c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:12.959105 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acceaf70-5182-49a4-819b-38ab575fdab9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3aa5a36fc084259b48466cd893e036c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:12.961025 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acceaf70-5182-49a4-819b-38ab575fdab9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:13.010297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-acceaf70-5182-49a4-819b-38ab575fdab9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3aa5a36fc084259b48466cd893e036c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:13.017559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-acceaf70-5182-49a4-819b-38ab575fdab9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3aa5a36fc084259b48466cd893e036c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:13.018355 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-acceaf70-5182-49a4-819b-38ab575fdab9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3aa5a36fc084259b48466cd893e036c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:13.026442 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-acceaf70-5182-49a4-819b-38ab575fdab9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3aa5a36fc084259b48466cd893e036c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:13.038753 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-acceaf70-5182-49a4-819b-38ab575fdab9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3aa5a36fc084259b48466cd893e036c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:13.045581 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1049/2124] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:33:12 2022] DELETE /identity/v3/users/d3aa5a36fc084259b48466cd893e036c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:13.049966 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4802874-0b48-4f88-b5ce-2a1808bcd7a6 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:13.050338 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4802874-0b48-4f88-b5ce-2a1808bcd7a6 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:13.050730 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4802874-0b48-4f88-b5ce-2a1808bcd7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:13.072130 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d4802874-0b48-4f88-b5ce-2a1808bcd7a6 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:13.074076 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d4802874-0b48-4f88-b5ce-2a1808bcd7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:13.133617 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1076/2125] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:33:13.143859 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acceaf70-5182-49a4-819b-38ab575fdab9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:13.145411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-acceaf70-5182-49a4-819b-38ab575fdab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:13.203410 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acceaf70-5182-49a4-819b-38ab575fdab9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:13.207888 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd35f8-00ec-452e-b53f-32b4cf788e7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:13.208496 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd35f8-00ec-452e-b53f-32b4cf788e7a service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:13.208781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd35f8-00ec-452e-b53f-32b4cf788e7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:13.209059 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd35f8-00ec-452e-b53f-32b4cf788e7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:13.218663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd35f8-00ec-452e-b53f-32b4cf788e7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:13.220093 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd35f8-00ec-452e-b53f-32b4cf788e7a service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:13.229442 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1050/2126] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:13.506033 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4802874-0b48-4f88-b5ce-2a1808bcd7a6 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:13.507867 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d4802874-0b48-4f88-b5ce-2a1808bcd7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:13.521237 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4802874-0b48-4f88-b5ce-2a1808bcd7a6 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:13.525794 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b287ce9a-b9d6-4bbf-8b83-97cd1bf6062e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:13.526454 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287ce9a-b9d6-4bbf-8b83-97cd1bf6062e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:13.526735 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287ce9a-b9d6-4bbf-8b83-97cd1bf6062e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:13.527008 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287ce9a-b9d6-4bbf-8b83-97cd1bf6062e admin admin] PATH_INFO: `/v3/projects/3dcbfb9669934bbcaaceb6f43dd44f5e` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:13.528117 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b287ce9a-b9d6-4bbf-8b83-97cd1bf6062e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dcbfb9669934bbcaaceb6f43dd44f5e)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:13.530016 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b287ce9a-b9d6-4bbf-8b83-97cd1bf6062e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:13.584526 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b287ce9a-b9d6-4bbf-8b83-97cd1bf6062e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dcbfb9669934bbcaaceb6f43dd44f5e', 'request_id': 'req-b287ce9a-b9d6-4bbf-8b83-97cd1bf6062e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:13.590932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b287ce9a-b9d6-4bbf-8b83-97cd1bf6062e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dcbfb9669934bbcaaceb6f43dd44f5e', 'request_id': 'req-b287ce9a-b9d6-4bbf-8b83-97cd1bf6062e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:13.596925 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b287ce9a-b9d6-4bbf-8b83-97cd1bf6062e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dcbfb9669934bbcaaceb6f43dd44f5e', 'request_id': 'req-b287ce9a-b9d6-4bbf-8b83-97cd1bf6062e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:13.610559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b287ce9a-b9d6-4bbf-8b83-97cd1bf6062e admin admin] The token cache is being invalidate because project 3dcbfb9669934bbcaaceb6f43dd44f5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:33:13.611004 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b287ce9a-b9d6-4bbf-8b83-97cd1bf6062e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:13.611700 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b287ce9a-b9d6-4bbf-8b83-97cd1bf6062e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:13.613222 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1077/2127] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:33:13 2022] DELETE /identity/v3/projects/3dcbfb9669934bbcaaceb6f43dd44f5e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:13.618072 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34672cbf-83a7-48df-89c5-6d3043afbf7c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:13.618560 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34672cbf-83a7-48df-89c5-6d3043afbf7c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:13.618651 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34672cbf-83a7-48df-89c5-6d3043afbf7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:13.643684 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-34672cbf-83a7-48df-89c5-6d3043afbf7c None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:13.644963 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-34672cbf-83a7-48df-89c5-6d3043afbf7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:13.665322 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:13.710713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1051/2128] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:13.718996 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff689b8a-dddb-4eba-b920-ed33e22dedbe None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:13.719364 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff689b8a-dddb-4eba-b920-ed33e22dedbe None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:13.719746 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff689b8a-dddb-4eba-b920-ed33e22dedbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:13.742052 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ff689b8a-dddb-4eba-b920-ed33e22dedbe None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:13.743419 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ff689b8a-dddb-4eba-b920-ed33e22dedbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:13.751894 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1078/2129] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:13.756115 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34672cbf-83a7-48df-89c5-6d3043afbf7c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:13.767017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34672cbf-83a7-48df-89c5-6d3043afbf7c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:13.772028 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8992955-793d-4c63-a090-bc86fc7e872c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:13.772672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8992955-793d-4c63-a090-bc86fc7e872c admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:13.772951 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8992955-793d-4c63-a090-bc86fc7e872c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:13.773226 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8992955-793d-4c63-a090-bc86fc7e872c admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:13.773886 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8992955-793d-4c63-a090-bc86fc7e872c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:33:13.774217 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8992955-793d-4c63-a090-bc86fc7e872c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:13.776319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8992955-793d-4c63-a090-bc86fc7e872c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:13.792932 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1052/2130] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:33:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:33:13.797647 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff689b8a-dddb-4eba-b920-ed33e22dedbe None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:13.806997 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff689b8a-dddb-4eba-b920-ed33e22dedbe None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:13.811617 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14cf94-7b83-43d4-a96c-e17ccd419317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:13.812211 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14cf94-7b83-43d4-a96c-e17ccd419317 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:13.812489 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14cf94-7b83-43d4-a96c-e17ccd419317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:13.812760 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14cf94-7b83-43d4-a96c-e17ccd419317 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:13.813400 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c14cf94-7b83-43d4-a96c-e17ccd419317 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:13.815322 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c14cf94-7b83-43d4-a96c-e17ccd419317 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:13.842842 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1079/2131] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:13 2022] POST /identity/v3/projects => generated 403 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:33:13.847051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8992955-793d-4c63-a090-bc86fc7e872c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:13.856865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8992955-793d-4c63-a090-bc86fc7e872c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:13.916055 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35952c8b-ab12-4c72-9998-6f41ae5a7827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:13.916689 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35952c8b-ab12-4c72-9998-6f41ae5a7827 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:13.916970 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35952c8b-ab12-4c72-9998-6f41ae5a7827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:13.917245 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35952c8b-ab12-4c72-9998-6f41ae5a7827 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:13.917889 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35952c8b-ab12-4c72-9998-6f41ae5a7827 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:13.920194 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35952c8b-ab12-4c72-9998-6f41ae5a7827 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:13.940423 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1053/2132] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:13 2022] POST /identity/v3/users => generated 302 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:13.944574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14cf94-7b83-43d4-a96c-e17ccd419317 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:13.955214 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14cf94-7b83-43d4-a96c-e17ccd419317 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:13.959863 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeddf74e-1e40-44fc-80f8-3d02bb78d3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:13.960455 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeddf74e-1e40-44fc-80f8-3d02bb78d3f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:13.960738 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeddf74e-1e40-44fc-80f8-3d02bb78d3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:13.961012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeddf74e-1e40-44fc-80f8-3d02bb78d3f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:13.961658 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeddf74e-1e40-44fc-80f8-3d02bb78d3f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:13.963891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeddf74e-1e40-44fc-80f8-3d02bb78d3f0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:13.975226 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1080/2133] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:13.979117 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35952c8b-ab12-4c72-9998-6f41ae5a7827 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:13.989215 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35952c8b-ab12-4c72-9998-6f41ae5a7827 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:13.994018 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14178416-72c3-47f5-8a7a-7134a806fa9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:13.995581 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14178416-72c3-47f5-8a7a-7134a806fa9b admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:13.995869 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14178416-72c3-47f5-8a7a-7134a806fa9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:13.996143 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14178416-72c3-47f5-8a7a-7134a806fa9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:13.996803 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14178416-72c3-47f5-8a7a-7134a806fa9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:13.998413 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14178416-72c3-47f5-8a7a-7134a806fa9b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:14.010938 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1054/2134] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:14.015183 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeddf74e-1e40-44fc-80f8-3d02bb78d3f0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:14.025151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeddf74e-1e40-44fc-80f8-3d02bb78d3f0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:14.030515 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bead41c-64a3-4899-a69a-807697566a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:14.031156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bead41c-64a3-4899-a69a-807697566a3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:14.031433 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bead41c-64a3-4899-a69a-807697566a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:14.031703 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bead41c-64a3-4899-a69a-807697566a3c admin admin] PATH_INFO: `/v3/projects/66ad21b827e24c13b8c2bbe8fc139a08/users/1f15ea00f11c452697b1572a59391d0c/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:14.048959 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bead41c-64a3-4899-a69a-807697566a3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=66ad21b827e24c13b8c2bbe8fc139a08, user_id=1f15ea00f11c452697b1572a59391d0c, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:14.051724 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bead41c-64a3-4899-a69a-807697566a3c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:14.065975 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1081/2135] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:14 2022] PUT /identity/v3/projects/66ad21b827e24c13b8c2bbe8fc139a08/users/1f15ea00f11c452697b1572a59391d0c/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:14.077216 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399fa6b3-c16a-4981-a6c9-0d1468788bc5 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:14.077500 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399fa6b3-c16a-4981-a6c9-0d1468788bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:14.077768 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399fa6b3-c16a-4981-a6c9-0d1468788bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:14.106106 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-399fa6b3-c16a-4981-a6c9-0d1468788bc5 None None] MFA Rules not processed for user `1f15ea00f11c452697b1572a59391d0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:14.107264 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-399fa6b3-c16a-4981-a6c9-0d1468788bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:14.174006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1055/2136] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:14 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:14.178460 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bead41c-64a3-4899-a69a-807697566a3c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:14.200608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:14.243107 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bead41c-64a3-4899-a69a-807697566a3c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:14.248479 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d93aba-93a8-4027-bc73-02b45ab39883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:14.249068 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d93aba-93a8-4027-bc73-02b45ab39883 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:14.249350 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d93aba-93a8-4027-bc73-02b45ab39883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:14.249622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d93aba-93a8-4027-bc73-02b45ab39883 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:14.250297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d93aba-93a8-4027-bc73-02b45ab39883 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:14.252209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d93aba-93a8-4027-bc73-02b45ab39883 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:14.283456 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1082/2137] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:14 2022] POST /identity/v3/projects => generated 401 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:14.285309 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399fa6b3-c16a-4981-a6c9-0d1468788bc5 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:14.307710 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399fa6b3-c16a-4981-a6c9-0d1468788bc5 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:14.322462 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:14.370854 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c18184-2e4b-4c7d-9390-414ab1d5c03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:14.371466 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c18184-2e4b-4c7d-9390-414ab1d5c03e admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:14.371753 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c18184-2e4b-4c7d-9390-414ab1d5c03e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:14.372030 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c18184-2e4b-4c7d-9390-414ab1d5c03e admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:14.372663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c18184-2e4b-4c7d-9390-414ab1d5c03e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:14.374694 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c18184-2e4b-4c7d-9390-414ab1d5c03e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:14.395577 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1056/2138] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:14 2022] POST /identity/v3/users => generated 307 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:14.399510 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d93aba-93a8-4027-bc73-02b45ab39883 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:14.410576 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d93aba-93a8-4027-bc73-02b45ab39883 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:14.415204 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e02bf-d0f0-49f1-b53f-ddcf09c9a8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:14.415802 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e02bf-d0f0-49f1-b53f-ddcf09c9a8ba admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:14.416084 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e02bf-d0f0-49f1-b53f-ddcf09c9a8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:14.416412 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e02bf-d0f0-49f1-b53f-ddcf09c9a8ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:14.417562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e02bf-d0f0-49f1-b53f-ddcf09c9a8ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:14.419201 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e02bf-d0f0-49f1-b53f-ddcf09c9a8ba admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:14.429576 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1083/2139] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:14.433605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c18184-2e4b-4c7d-9390-414ab1d5c03e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:14.443592 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c18184-2e4b-4c7d-9390-414ab1d5c03e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:14.448336 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1bde7c-3a86-4c39-82a1-4cf044e3ccfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:14.448975 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bde7c-3a86-4c39-82a1-4cf044e3ccfc admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:14.449259 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bde7c-3a86-4c39-82a1-4cf044e3ccfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:14.449532 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bde7c-3a86-4c39-82a1-4cf044e3ccfc admin admin] PATH_INFO: `/v3/projects/b63c188636a543b9a8b4a618ed56e0ee/users/5e03368c44034084a1d1a0bf11a80221/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:14.464456 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1bde7c-3a86-4c39-82a1-4cf044e3ccfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b63c188636a543b9a8b4a618ed56e0ee, user_id=5e03368c44034084a1d1a0bf11a80221, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:14.467130 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1bde7c-3a86-4c39-82a1-4cf044e3ccfc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:14.480213 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1057/2140] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:14 2022] PUT /identity/v3/projects/b63c188636a543b9a8b4a618ed56e0ee/users/5e03368c44034084a1d1a0bf11a80221/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:14.484205 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e02bf-d0f0-49f1-b53f-ddcf09c9a8ba admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:14.494048 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e02bf-d0f0-49f1-b53f-ddcf09c9a8ba admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:14.551023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4854e7-1506-43e8-a38d-372e71843ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:14.551665 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4854e7-1506-43e8-a38d-372e71843ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:14.551892 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4854e7-1506-43e8-a38d-372e71843ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:14.552163 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4854e7-1506-43e8-a38d-372e71843ef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:14.552799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4854e7-1506-43e8-a38d-372e71843ef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:14.554404 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4854e7-1506-43e8-a38d-372e71843ef3 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:14.563955 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1084/2141] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:14 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:14.568039 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1bde7c-3a86-4c39-82a1-4cf044e3ccfc admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:14.579203 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1bde7c-3a86-4c39-82a1-4cf044e3ccfc admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:14.583758 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f060d86e-84a4-4d6c-801d-a8433e73351e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:14.584405 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f060d86e-84a4-4d6c-801d-a8433e73351e admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:14.584681 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f060d86e-84a4-4d6c-801d-a8433e73351e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:14.584982 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f060d86e-84a4-4d6c-801d-a8433e73351e admin admin] PATH_INFO: `/v3/projects/b63c188636a543b9a8b4a618ed56e0ee/users/5e03368c44034084a1d1a0bf11a80221/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:14.586847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f060d86e-84a4-4d6c-801d-a8433e73351e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b63c188636a543b9a8b4a618ed56e0ee, user_id=5e03368c44034084a1d1a0bf11a80221, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:14.590130 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f060d86e-84a4-4d6c-801d-a8433e73351e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:14.600095 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1058/2142] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:14 2022] PUT /identity/v3/projects/b63c188636a543b9a8b4a618ed56e0ee/users/5e03368c44034084a1d1a0bf11a80221/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:14.611299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf955ac-b3ee-4d9c-8c2f-70abb7255e13 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:14.611584 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf955ac-b3ee-4d9c-8c2f-70abb7255e13 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:14.611859 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf955ac-b3ee-4d9c-8c2f-70abb7255e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:14.639736 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5bf955ac-b3ee-4d9c-8c2f-70abb7255e13 None None] MFA Rules not processed for user `5e03368c44034084a1d1a0bf11a80221`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:14.640855 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5bf955ac-b3ee-4d9c-8c2f-70abb7255e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:14.706372 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1085/2143] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:14 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:33:14.717582 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f060d86e-84a4-4d6c-801d-a8433e73351e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:14.719326 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f060d86e-84a4-4d6c-801d-a8433e73351e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:14.775189 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f060d86e-84a4-4d6c-801d-a8433e73351e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:14.779439 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd0967-8086-4c20-8fcc-e05809d72d12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:14.780033 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd0967-8086-4c20-8fcc-e05809d72d12 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:14.780311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd0967-8086-4c20-8fcc-e05809d72d12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:14.780585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd0967-8086-4c20-8fcc-e05809d72d12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:14.789578 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cd0967-8086-4c20-8fcc-e05809d72d12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:14.791042 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cd0967-8086-4c20-8fcc-e05809d72d12 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:14.800378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1059/2144] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:14 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:14.955761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf955ac-b3ee-4d9c-8c2f-70abb7255e13 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:14.973408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:15.021357 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf955ac-b3ee-4d9c-8c2f-70abb7255e13 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:15.025875 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31788ba4-427a-42b0-8fb4-ab34aa112986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:15.026528 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31788ba4-427a-42b0-8fb4-ab34aa112986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:15.026808 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31788ba4-427a-42b0-8fb4-ab34aa112986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:15.027081 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31788ba4-427a-42b0-8fb4-ab34aa112986 admin admin] PATH_INFO: `/v3/users/1f15ea00f11c452697b1572a59391d0c` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:15.028204 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31788ba4-427a-42b0-8fb4-ab34aa112986 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f15ea00f11c452697b1572a59391d0c)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:15.030117 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31788ba4-427a-42b0-8fb4-ab34aa112986 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:15.082888 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-31788ba4-427a-42b0-8fb4-ab34aa112986 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f15ea00f11c452697b1572a59391d0c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:15.090022 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-31788ba4-427a-42b0-8fb4-ab34aa112986 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f15ea00f11c452697b1572a59391d0c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:15.090723 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-31788ba4-427a-42b0-8fb4-ab34aa112986 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f15ea00f11c452697b1572a59391d0c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:15.091324 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-31788ba4-427a-42b0-8fb4-ab34aa112986 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f15ea00f11c452697b1572a59391d0c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:15.103256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-31788ba4-427a-42b0-8fb4-ab34aa112986 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f15ea00f11c452697b1572a59391d0c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:15.118274 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1086/2145] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:33:14 2022] DELETE /identity/v3/users/1f15ea00f11c452697b1572a59391d0c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:15.123645 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9b418-082b-40c5-b1ac-404abbc0db3a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:15.124001 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9b418-082b-40c5-b1ac-404abbc0db3a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:15.124429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9b418-082b-40c5-b1ac-404abbc0db3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:15.143965 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7cf9b418-082b-40c5-b1ac-404abbc0db3a None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:15.145170 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7cf9b418-082b-40c5-b1ac-404abbc0db3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:15.205517 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1060/2146] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:15.214980 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31788ba4-427a-42b0-8fb4-ab34aa112986 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:15.216498 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-31788ba4-427a-42b0-8fb4-ab34aa112986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:15.276159 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31788ba4-427a-42b0-8fb4-ab34aa112986 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:15.280212 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef54f7-05f2-4942-940a-70ca0898182a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:15.280821 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef54f7-05f2-4942-940a-70ca0898182a service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:15.281094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef54f7-05f2-4942-940a-70ca0898182a service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:15.281370 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef54f7-05f2-4942-940a-70ca0898182a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:15.290888 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef54f7-05f2-4942-940a-70ca0898182a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:15.292822 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef54f7-05f2-4942-940a-70ca0898182a service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:15.302471 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1087/2147] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:15.794045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf9b418-082b-40c5-b1ac-404abbc0db3a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:15.795614 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7cf9b418-082b-40c5-b1ac-404abbc0db3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:15.808248 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf9b418-082b-40c5-b1ac-404abbc0db3a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:15.813335 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e4989-519d-4b01-b67c-12dcbc735d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:15.814277 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e4989-519d-4b01-b67c-12dcbc735d5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:15.814514 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e4989-519d-4b01-b67c-12dcbc735d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:15.814943 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e4989-519d-4b01-b67c-12dcbc735d5a admin admin] PATH_INFO: `/v3/projects/66ad21b827e24c13b8c2bbe8fc139a08` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:15.816212 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e4989-519d-4b01-b67c-12dcbc735d5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=66ad21b827e24c13b8c2bbe8fc139a08)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:15.818142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e4989-519d-4b01-b67c-12dcbc735d5a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:15.826810 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:15.867563 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5e9e4989-519d-4b01-b67c-12dcbc735d5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66ad21b827e24c13b8c2bbe8fc139a08', 'request_id': 'req-5e9e4989-519d-4b01-b67c-12dcbc735d5a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:15.873680 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5e9e4989-519d-4b01-b67c-12dcbc735d5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66ad21b827e24c13b8c2bbe8fc139a08', 'request_id': 'req-5e9e4989-519d-4b01-b67c-12dcbc735d5a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:15.881157 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5e9e4989-519d-4b01-b67c-12dcbc735d5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66ad21b827e24c13b8c2bbe8fc139a08', 'request_id': 'req-5e9e4989-519d-4b01-b67c-12dcbc735d5a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:15.891424 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5e9e4989-519d-4b01-b67c-12dcbc735d5a admin admin] The token cache is being invalidate because project 66ad21b827e24c13b8c2bbe8fc139a08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:33:15.891710 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5e9e4989-519d-4b01-b67c-12dcbc735d5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:15.892328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5e9e4989-519d-4b01-b67c-12dcbc735d5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:15.893790 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1061/2148] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:33:15 2022] DELETE /identity/v3/projects/66ad21b827e24c13b8c2bbe8fc139a08 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:15.897892 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef54f7-05f2-4942-940a-70ca0898182a service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:15.909811 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef54f7-05f2-4942-940a-70ca0898182a service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:15.943352 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:15.967981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d558a31-cb6f-4059-b63b-56fa4a6ad447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:15.968540 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d558a31-cb6f-4059-b63b-56fa4a6ad447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:15.968803 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d558a31-cb6f-4059-b63b-56fa4a6ad447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:15.969080 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d558a31-cb6f-4059-b63b-56fa4a6ad447 admin admin] PATH_INFO: `/v3/users/5e03368c44034084a1d1a0bf11a80221` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:15.970187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d558a31-cb6f-4059-b63b-56fa4a6ad447 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e03368c44034084a1d1a0bf11a80221)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:15.972118 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d558a31-cb6f-4059-b63b-56fa4a6ad447 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:16.022843 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0d558a31-cb6f-4059-b63b-56fa4a6ad447 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e03368c44034084a1d1a0bf11a80221'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:16.036198 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0d558a31-cb6f-4059-b63b-56fa4a6ad447 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e03368c44034084a1d1a0bf11a80221'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:16.036930 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0d558a31-cb6f-4059-b63b-56fa4a6ad447 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e03368c44034084a1d1a0bf11a80221'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:16.037958 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0d558a31-cb6f-4059-b63b-56fa4a6ad447 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e03368c44034084a1d1a0bf11a80221'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:16.061958 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0d558a31-cb6f-4059-b63b-56fa4a6ad447 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e03368c44034084a1d1a0bf11a80221'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:16.081613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1088/2149] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:33:15 2022] DELETE /identity/v3/users/5e03368c44034084a1d1a0bf11a80221 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:16.356379 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e4989-519d-4b01-b67c-12dcbc735d5a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:16.357868 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5e9e4989-519d-4b01-b67c-12dcbc735d5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:16.372098 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e4989-519d-4b01-b67c-12dcbc735d5a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:16.430578 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf96a1-f120-4987-8f07-ff5c362b595c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:16.431212 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf96a1-f120-4987-8f07-ff5c362b595c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:16.431494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf96a1-f120-4987-8f07-ff5c362b595c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:16.431769 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf96a1-f120-4987-8f07-ff5c362b595c admin admin] PATH_INFO: `/v3/projects/b63c188636a543b9a8b4a618ed56e0ee` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:16.433384 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf96a1-f120-4987-8f07-ff5c362b595c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b63c188636a543b9a8b4a618ed56e0ee)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:16.435336 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf96a1-f120-4987-8f07-ff5c362b595c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:16.481476 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-42bf96a1-f120-4987-8f07-ff5c362b595c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b63c188636a543b9a8b4a618ed56e0ee', 'request_id': 'req-42bf96a1-f120-4987-8f07-ff5c362b595c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:16.487502 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-42bf96a1-f120-4987-8f07-ff5c362b595c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b63c188636a543b9a8b4a618ed56e0ee', 'request_id': 'req-42bf96a1-f120-4987-8f07-ff5c362b595c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:16.497797 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-42bf96a1-f120-4987-8f07-ff5c362b595c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b63c188636a543b9a8b4a618ed56e0ee', 'request_id': 'req-42bf96a1-f120-4987-8f07-ff5c362b595c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:16.516171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-42bf96a1-f120-4987-8f07-ff5c362b595c admin admin] The token cache is being invalidate because project b63c188636a543b9a8b4a618ed56e0ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:33:16.516470 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-42bf96a1-f120-4987-8f07-ff5c362b595c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:16.517147 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-42bf96a1-f120-4987-8f07-ff5c362b595c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:16.518664 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1062/2150] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:33:16 2022] DELETE /identity/v3/projects/b63c188636a543b9a8b4a618ed56e0ee => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:16.523495 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100da33-00fd-4fee-88e7-56ac2702c903 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:16.523779 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100da33-00fd-4fee-88e7-56ac2702c903 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:16.524087 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100da33-00fd-4fee-88e7-56ac2702c903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:16.552503 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-8100da33-00fd-4fee-88e7-56ac2702c903 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:16.554148 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8100da33-00fd-4fee-88e7-56ac2702c903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:16.623140 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1089/2151] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:16.630803 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f706ac-b2f2-4d69-8562-965d127b9092 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:16.631181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f706ac-b2f2-4d69-8562-965d127b9092 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:16.631513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f706ac-b2f2-4d69-8562-965d127b9092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:16.653777 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-28f706ac-b2f2-4d69-8562-965d127b9092 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:16.655028 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-28f706ac-b2f2-4d69-8562-965d127b9092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:16.668133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1063/2152] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:16.669312 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8100da33-00fd-4fee-88e7-56ac2702c903 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:16.680898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8100da33-00fd-4fee-88e7-56ac2702c903 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:16.686224 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4fe3ee-1286-4e86-b0c5-30c4027e8e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:16.686847 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4fe3ee-1286-4e86-b0c5-30c4027e8e2a admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:16.687133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4fe3ee-1286-4e86-b0c5-30c4027e8e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:16.687835 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4fe3ee-1286-4e86-b0c5-30c4027e8e2a admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:16.688057 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4fe3ee-1286-4e86-b0c5-30c4027e8e2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:33:16.688364 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4fe3ee-1286-4e86-b0c5-30c4027e8e2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:16.689959 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4fe3ee-1286-4e86-b0c5-30c4027e8e2a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:16.703698 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1090/2153] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:33:16.708437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f706ac-b2f2-4d69-8562-965d127b9092 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:16.718632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f706ac-b2f2-4d69-8562-965d127b9092 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:16.725755 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492beb5-153d-4565-9aa4-254871ede50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:16.726517 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492beb5-153d-4565-9aa4-254871ede50d admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:16.726813 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492beb5-153d-4565-9aa4-254871ede50d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:16.727253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492beb5-153d-4565-9aa4-254871ede50d admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:16.727921 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5492beb5-153d-4565-9aa4-254871ede50d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:16.730722 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5492beb5-153d-4565-9aa4-254871ede50d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:16.755340 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:16.768878 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1064/2154] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:16 2022] POST /identity/v3/projects => generated 367 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:16.771347 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4fe3ee-1286-4e86-b0c5-30c4027e8e2a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:16.781040 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4fe3ee-1286-4e86-b0c5-30c4027e8e2a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:16.834311 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:16.857577 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb5c20-5b9c-4fc3-bc7a-a82fc23e4f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:16.858212 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb5c20-5b9c-4fc3-bc7a-a82fc23e4f43 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:16.858495 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb5c20-5b9c-4fc3-bc7a-a82fc23e4f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:16.858763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb5c20-5b9c-4fc3-bc7a-a82fc23e4f43 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:16.859414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb5c20-5b9c-4fc3-bc7a-a82fc23e4f43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:16.861306 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb5c20-5b9c-4fc3-bc7a-a82fc23e4f43 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:16.886446 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1091/2155] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:16 2022] POST /identity/v3/users => generated 284 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:16.887866 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492beb5-153d-4565-9aa4-254871ede50d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:16.904410 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492beb5-153d-4565-9aa4-254871ede50d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:16.910843 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c5019-89cf-4777-a633-6570f982b3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:16.911522 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c5019-89cf-4777-a633-6570f982b3cf admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:16.911740 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c5019-89cf-4777-a633-6570f982b3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:16.912017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c5019-89cf-4777-a633-6570f982b3cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:16.912682 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c5019-89cf-4777-a633-6570f982b3cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:16.914335 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c5019-89cf-4777-a633-6570f982b3cf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:16.925947 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1065/2156] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:16.929883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb5c20-5b9c-4fc3-bc7a-a82fc23e4f43 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:16.940413 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb5c20-5b9c-4fc3-bc7a-a82fc23e4f43 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:16.945216 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9e2b1-888f-4fb0-b28f-e3bd754bb265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:16.945889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9e2b1-888f-4fb0-b28f-e3bd754bb265 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:16.946272 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9e2b1-888f-4fb0-b28f-e3bd754bb265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:16.946558 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9e2b1-888f-4fb0-b28f-e3bd754bb265 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:16.947208 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf9e2b1-888f-4fb0-b28f-e3bd754bb265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:16.949438 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf9e2b1-888f-4fb0-b28f-e3bd754bb265 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:16.964135 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1092/2157] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:16.965315 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c5019-89cf-4777-a633-6570f982b3cf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:16.975236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c5019-89cf-4777-a633-6570f982b3cf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:16.979978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145872d8-30b5-446e-8ae4-adf6223a756e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:16.980631 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145872d8-30b5-446e-8ae4-adf6223a756e admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:16.980917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145872d8-30b5-446e-8ae4-adf6223a756e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:16.981199 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145872d8-30b5-446e-8ae4-adf6223a756e admin admin] PATH_INFO: `/v3/projects/20dc07d0909a4a57abe49ee3f42f4880/users/ba0d023559ac4b6ba7deda044e3fa97d/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:16.995773 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145872d8-30b5-446e-8ae4-adf6223a756e admin admin] RBAC: Authorizing `identity:create_grant(project_id=20dc07d0909a4a57abe49ee3f42f4880, user_id=ba0d023559ac4b6ba7deda044e3fa97d, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:16.998400 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145872d8-30b5-446e-8ae4-adf6223a756e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:17.015340 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1066/2158] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:16 2022] PUT /identity/v3/projects/20dc07d0909a4a57abe49ee3f42f4880/users/ba0d023559ac4b6ba7deda044e3fa97d/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:17.025693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6aa226-f785-427d-9328-431e6cc05930 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:17.026505 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6aa226-f785-427d-9328-431e6cc05930 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:17.026778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6aa226-f785-427d-9328-431e6cc05930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:17.055314 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ab6aa226-f785-427d-9328-431e6cc05930 None None] MFA Rules not processed for user `ba0d023559ac4b6ba7deda044e3fa97d`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:17.056515 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ab6aa226-f785-427d-9328-431e6cc05930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:17.119846 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1093/2159] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:17.124347 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145872d8-30b5-446e-8ae4-adf6223a756e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:17.186509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145872d8-30b5-446e-8ae4-adf6223a756e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:17.191230 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58129e41-122d-48ca-b1c2-f43d3e396f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:17.191847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58129e41-122d-48ca-b1c2-f43d3e396f56 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:17.192133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58129e41-122d-48ca-b1c2-f43d3e396f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:17.192407 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58129e41-122d-48ca-b1c2-f43d3e396f56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:17.193042 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58129e41-122d-48ca-b1c2-f43d3e396f56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:17.194981 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58129e41-122d-48ca-b1c2-f43d3e396f56 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:17.223384 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1067/2160] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:17 2022] POST /identity/v3/projects => generated 369 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:17.227388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6aa226-f785-427d-9328-431e6cc05930 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:17.238119 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6aa226-f785-427d-9328-431e6cc05930 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:17.296197 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b99c2-ae86-45dd-b0e5-2c221ffef6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:17.296802 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b99c2-ae86-45dd-b0e5-2c221ffef6c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:17.297079 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b99c2-ae86-45dd-b0e5-2c221ffef6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:17.297351 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b99c2-ae86-45dd-b0e5-2c221ffef6c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:17.297988 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b99c2-ae86-45dd-b0e5-2c221ffef6c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:17.299917 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b99c2-ae86-45dd-b0e5-2c221ffef6c2 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:17.325066 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1094/2161] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:17 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:17.326423 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58129e41-122d-48ca-b1c2-f43d3e396f56 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:17.337182 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58129e41-122d-48ca-b1c2-f43d3e396f56 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:17.341926 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030afe5f-bec4-451a-8c0b-749f886de961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:17.342668 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030afe5f-bec4-451a-8c0b-749f886de961 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:17.342953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030afe5f-bec4-451a-8c0b-749f886de961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:17.343232 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030afe5f-bec4-451a-8c0b-749f886de961 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:17.343916 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030afe5f-bec4-451a-8c0b-749f886de961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:17.345512 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030afe5f-bec4-451a-8c0b-749f886de961 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:17.357431 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1068/2162] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:17.361284 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b99c2-ae86-45dd-b0e5-2c221ffef6c2 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:17.371820 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b99c2-ae86-45dd-b0e5-2c221ffef6c2 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:17.376385 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc9922b-299e-4503-b167-73ca4577d060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:17.377030 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9922b-299e-4503-b167-73ca4577d060 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:17.377309 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9922b-299e-4503-b167-73ca4577d060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:17.377580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9922b-299e-4503-b167-73ca4577d060 admin admin] PATH_INFO: `/v3/projects/670c1e89960d459b8546dac7df0ac0d6/users/683ae9c47fd94a0d94cf7eb91616774a/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:17.394917 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc9922b-299e-4503-b167-73ca4577d060 admin admin] RBAC: Authorizing `identity:create_grant(project_id=670c1e89960d459b8546dac7df0ac0d6, user_id=683ae9c47fd94a0d94cf7eb91616774a, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:17.397613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc9922b-299e-4503-b167-73ca4577d060 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:17.413059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1095/2163] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:17 2022] PUT /identity/v3/projects/670c1e89960d459b8546dac7df0ac0d6/users/683ae9c47fd94a0d94cf7eb91616774a/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:17.417625 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030afe5f-bec4-451a-8c0b-749f886de961 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:17.427576 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030afe5f-bec4-451a-8c0b-749f886de961 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:17.485854 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a50b0-afc9-4932-a262-ba87172887a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:17.486504 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a50b0-afc9-4932-a262-ba87172887a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:17.486792 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a50b0-afc9-4932-a262-ba87172887a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:17.487072 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a50b0-afc9-4932-a262-ba87172887a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:17.487731 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a50b0-afc9-4932-a262-ba87172887a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:17.489328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a50b0-afc9-4932-a262-ba87172887a1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:17.499467 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1069/2164] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:17.503667 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc9922b-299e-4503-b167-73ca4577d060 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:17.515103 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc9922b-299e-4503-b167-73ca4577d060 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:17.520172 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487c037-ff86-49f7-8896-c386829f0b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:17.520931 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487c037-ff86-49f7-8896-c386829f0b98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:17.521215 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487c037-ff86-49f7-8896-c386829f0b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:17.521493 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487c037-ff86-49f7-8896-c386829f0b98 admin admin] PATH_INFO: `/v3/projects/670c1e89960d459b8546dac7df0ac0d6/users/683ae9c47fd94a0d94cf7eb91616774a/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:17.523724 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487c037-ff86-49f7-8896-c386829f0b98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=670c1e89960d459b8546dac7df0ac0d6, user_id=683ae9c47fd94a0d94cf7eb91616774a, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:17.527110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487c037-ff86-49f7-8896-c386829f0b98 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:17.537822 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1096/2165] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:17 2022] PUT /identity/v3/projects/670c1e89960d459b8546dac7df0ac0d6/users/683ae9c47fd94a0d94cf7eb91616774a/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:17.549708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1577989-334d-4e01-9970-06e35efd60b4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:17.550072 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1577989-334d-4e01-9970-06e35efd60b4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:17.550513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1577989-334d-4e01-9970-06e35efd60b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:17.579530 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d1577989-334d-4e01-9970-06e35efd60b4 None None] MFA Rules not processed for user `683ae9c47fd94a0d94cf7eb91616774a`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:17.580734 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d1577989-334d-4e01-9970-06e35efd60b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:17.601781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:17.653560 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1070/2166] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:17 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:17.665220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487c037-ff86-49f7-8896-c386829f0b98 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:17.667625 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2487c037-ff86-49f7-8896-c386829f0b98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:17.674036 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:17.728032 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487c037-ff86-49f7-8896-c386829f0b98 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:17.732471 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70414d4-f46e-4a2b-bef7-7e43af4b4a2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:17.733079 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70414d4-f46e-4a2b-bef7-7e43af4b4a2d service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:17.733361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70414d4-f46e-4a2b-bef7-7e43af4b4a2d service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:17.733638 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70414d4-f46e-4a2b-bef7-7e43af4b4a2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:17.742839 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70414d4-f46e-4a2b-bef7-7e43af4b4a2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:17.744428 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70414d4-f46e-4a2b-bef7-7e43af4b4a2d service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:17.754238 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1097/2167] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:17 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:17.800103 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1577989-334d-4e01-9970-06e35efd60b4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:17.801471 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d1577989-334d-4e01-9970-06e35efd60b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:17.813802 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1577989-334d-4e01-9970-06e35efd60b4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:17.817788 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035e008-1c3c-4803-97a7-3f786ca7e524 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:17.818418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035e008-1c3c-4803-97a7-3f786ca7e524 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:17.818704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035e008-1c3c-4803-97a7-3f786ca7e524 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:17.818973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035e008-1c3c-4803-97a7-3f786ca7e524 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:17.875785 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d035e008-1c3c-4803-97a7-3f786ca7e524 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:17.877228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d035e008-1c3c-4803-97a7-3f786ca7e524 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:17.886037 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1071/2168] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:21.196891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70414d4-f46e-4a2b-bef7-7e43af4b4a2d service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:21.263342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70414d4-f46e-4a2b-bef7-7e43af4b4a2d service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:21.267909 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c15a4-dcee-477e-a0c7-a0e458338e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:21.268622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c15a4-dcee-477e-a0c7-a0e458338e9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:21.268908 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c15a4-dcee-477e-a0c7-a0e458338e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:21.269179 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c15a4-dcee-477e-a0c7-a0e458338e9f admin admin] PATH_INFO: `/v3/users/ba0d023559ac4b6ba7deda044e3fa97d` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:21.270426 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9c15a4-dcee-477e-a0c7-a0e458338e9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba0d023559ac4b6ba7deda044e3fa97d)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:21.272472 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9c15a4-dcee-477e-a0c7-a0e458338e9f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:21.321592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ce9c15a4-dcee-477e-a0c7-a0e458338e9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba0d023559ac4b6ba7deda044e3fa97d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:21.328852 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ce9c15a4-dcee-477e-a0c7-a0e458338e9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba0d023559ac4b6ba7deda044e3fa97d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:21.329511 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ce9c15a4-dcee-477e-a0c7-a0e458338e9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba0d023559ac4b6ba7deda044e3fa97d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:21.330114 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ce9c15a4-dcee-477e-a0c7-a0e458338e9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba0d023559ac4b6ba7deda044e3fa97d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:21.342238 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ce9c15a4-dcee-477e-a0c7-a0e458338e9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba0d023559ac4b6ba7deda044e3fa97d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:21.355443 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1098/2169] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:33:21 2022] DELETE /identity/v3/users/ba0d023559ac4b6ba7deda044e3fa97d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:21.359946 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cbe42-0944-4376-8e1f-1530f07e0cb9 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:21.360502 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cbe42-0944-4376-8e1f-1530f07e0cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:21.360502 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cbe42-0944-4376-8e1f-1530f07e0cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:21.379221 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6d9cbe42-0944-4376-8e1f-1530f07e0cb9 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:21.380403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6d9cbe42-0944-4376-8e1f-1530f07e0cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:21.439750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1072/2170] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:21.450006 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c15a4-dcee-477e-a0c7-a0e458338e9f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:21.451475 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ce9c15a4-dcee-477e-a0c7-a0e458338e9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:21.482110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:21.509100 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c15a4-dcee-477e-a0c7-a0e458338e9f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:21.513277 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7512673-0abe-40a4-a98c-d8c79435a432 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:21.513879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7512673-0abe-40a4-a98c-d8c79435a432 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:21.514161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7512673-0abe-40a4-a98c-d8c79435a432 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:21.514466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7512673-0abe-40a4-a98c-d8c79435a432 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:21.523505 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7512673-0abe-40a4-a98c-d8c79435a432 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:21.524908 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7512673-0abe-40a4-a98c-d8c79435a432 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:21.535261 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1099/2171] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:21.856605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9cbe42-0944-4376-8e1f-1530f07e0cb9 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:21.858334 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6d9cbe42-0944-4376-8e1f-1530f07e0cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:21.874693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9cbe42-0944-4376-8e1f-1530f07e0cb9 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:21.880761 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4dc8e7-dac6-4c2c-a7d7-2117428017d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:21.881468 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4dc8e7-dac6-4c2c-a7d7-2117428017d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:21.881757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4dc8e7-dac6-4c2c-a7d7-2117428017d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:21.882036 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4dc8e7-dac6-4c2c-a7d7-2117428017d1 admin admin] PATH_INFO: `/v3/projects/20dc07d0909a4a57abe49ee3f42f4880` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:21.883214 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4dc8e7-dac6-4c2c-a7d7-2117428017d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20dc07d0909a4a57abe49ee3f42f4880)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:21.885935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4dc8e7-dac6-4c2c-a7d7-2117428017d1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:21.936472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3f4dc8e7-dac6-4c2c-a7d7-2117428017d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20dc07d0909a4a57abe49ee3f42f4880', 'request_id': 'req-3f4dc8e7-dac6-4c2c-a7d7-2117428017d1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:21.938633 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:21.943376 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3f4dc8e7-dac6-4c2c-a7d7-2117428017d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20dc07d0909a4a57abe49ee3f42f4880', 'request_id': 'req-3f4dc8e7-dac6-4c2c-a7d7-2117428017d1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:21.951929 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3f4dc8e7-dac6-4c2c-a7d7-2117428017d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20dc07d0909a4a57abe49ee3f42f4880', 'request_id': 'req-3f4dc8e7-dac6-4c2c-a7d7-2117428017d1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:21.965790 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3f4dc8e7-dac6-4c2c-a7d7-2117428017d1 admin admin] The token cache is being invalidate because project 20dc07d0909a4a57abe49ee3f42f4880 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:33:21.967203 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3f4dc8e7-dac6-4c2c-a7d7-2117428017d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:21.968273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3f4dc8e7-dac6-4c2c-a7d7-2117428017d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:21.970759 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1073/2172] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:33:21 2022] DELETE /identity/v3/projects/20dc07d0909a4a57abe49ee3f42f4880 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:21.974798 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7512673-0abe-40a4-a98c-d8c79435a432 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:21.985091 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7512673-0abe-40a4-a98c-d8c79435a432 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:22.053950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de77aef-53c4-449c-bd1c-97226519987b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:22.054638 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de77aef-53c4-449c-bd1c-97226519987b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:22.054914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de77aef-53c4-449c-bd1c-97226519987b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:22.055354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de77aef-53c4-449c-bd1c-97226519987b admin admin] PATH_INFO: `/v3/users/683ae9c47fd94a0d94cf7eb91616774a` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:22.057254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de77aef-53c4-449c-bd1c-97226519987b admin admin] RBAC: Authorizing `identity:delete_user(user_id=683ae9c47fd94a0d94cf7eb91616774a)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:22.059488 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de77aef-53c4-449c-bd1c-97226519987b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:22.114792 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0de77aef-53c4-449c-bd1c-97226519987b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '683ae9c47fd94a0d94cf7eb91616774a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:22.122506 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0de77aef-53c4-449c-bd1c-97226519987b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '683ae9c47fd94a0d94cf7eb91616774a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:22.123192 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0de77aef-53c4-449c-bd1c-97226519987b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '683ae9c47fd94a0d94cf7eb91616774a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:22.123805 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0de77aef-53c4-449c-bd1c-97226519987b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '683ae9c47fd94a0d94cf7eb91616774a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:22.135481 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0de77aef-53c4-449c-bd1c-97226519987b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '683ae9c47fd94a0d94cf7eb91616774a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:22.147962 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1100/2173] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:33:21 2022] DELETE /identity/v3/users/683ae9c47fd94a0d94cf7eb91616774a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:22.411655 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4dc8e7-dac6-4c2c-a7d7-2117428017d1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:22.413149 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3f4dc8e7-dac6-4c2c-a7d7-2117428017d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:22.433097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4dc8e7-dac6-4c2c-a7d7-2117428017d1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:22.491937 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962bfc6-9b49-459c-bf27-7496eaf8f6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:22.492500 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962bfc6-9b49-459c-bf27-7496eaf8f6ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:22.492784 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962bfc6-9b49-459c-bf27-7496eaf8f6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:22.493062 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962bfc6-9b49-459c-bf27-7496eaf8f6ee admin admin] PATH_INFO: `/v3/projects/670c1e89960d459b8546dac7df0ac0d6` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:22.494219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962bfc6-9b49-459c-bf27-7496eaf8f6ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=670c1e89960d459b8546dac7df0ac0d6)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:22.496136 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962bfc6-9b49-459c-bf27-7496eaf8f6ee admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:22.541973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e962bfc6-9b49-459c-bf27-7496eaf8f6ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '670c1e89960d459b8546dac7df0ac0d6', 'request_id': 'req-e962bfc6-9b49-459c-bf27-7496eaf8f6ee'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:22.547923 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e962bfc6-9b49-459c-bf27-7496eaf8f6ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '670c1e89960d459b8546dac7df0ac0d6', 'request_id': 'req-e962bfc6-9b49-459c-bf27-7496eaf8f6ee'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:22.555605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e962bfc6-9b49-459c-bf27-7496eaf8f6ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '670c1e89960d459b8546dac7df0ac0d6', 'request_id': 'req-e962bfc6-9b49-459c-bf27-7496eaf8f6ee'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:22.566347 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e962bfc6-9b49-459c-bf27-7496eaf8f6ee admin admin] The token cache is being invalidate because project 670c1e89960d459b8546dac7df0ac0d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:33:22.566636 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e962bfc6-9b49-459c-bf27-7496eaf8f6ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:22.567265 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e962bfc6-9b49-459c-bf27-7496eaf8f6ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:22.568788 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1074/2174] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:33:22 2022] DELETE /identity/v3/projects/670c1e89960d459b8546dac7df0ac0d6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:22.574475 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721aacc-5fe6-4325-b6f7-b5c3292f3ff5 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:22.574979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721aacc-5fe6-4325-b6f7-b5c3292f3ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:22.575166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721aacc-5fe6-4325-b6f7-b5c3292f3ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:22.595311 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5721aacc-5fe6-4325-b6f7-b5c3292f3ff5 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:22.596508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5721aacc-5fe6-4325-b6f7-b5c3292f3ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:22.642471 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:22.659108 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1101/2175] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:22.667481 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935b08c-29e5-424f-8a4d-1586ac23c1d8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:22.667791 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935b08c-29e5-424f-8a4d-1586ac23c1d8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:22.668093 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935b08c-29e5-424f-8a4d-1586ac23c1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:22.687493 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5935b08c-29e5-424f-8a4d-1586ac23c1d8 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:22.688741 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5935b08c-29e5-424f-8a4d-1586ac23c1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:22.696700 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1075/2176] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:22.700817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721aacc-5fe6-4325-b6f7-b5c3292f3ff5 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:22.710926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721aacc-5fe6-4325-b6f7-b5c3292f3ff5 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:22.716584 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b65b8-d833-4d76-9939-bdedcb65a445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:22.717185 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b65b8-d833-4d76-9939-bdedcb65a445 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:22.717458 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b65b8-d833-4d76-9939-bdedcb65a445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:22.717726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b65b8-d833-4d76-9939-bdedcb65a445 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:22.718594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42b65b8-d833-4d76-9939-bdedcb65a445 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:33:22.718898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42b65b8-d833-4d76-9939-bdedcb65a445 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:22.720487 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42b65b8-d833-4d76-9939-bdedcb65a445 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:22.733305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1102/2177] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:33:22.737734 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935b08c-29e5-424f-8a4d-1586ac23c1d8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:22.747313 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935b08c-29e5-424f-8a4d-1586ac23c1d8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:22.751862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e3dea-e91e-4a6f-8625-48f32925843e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:22.752454 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e3dea-e91e-4a6f-8625-48f32925843e admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:22.752734 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e3dea-e91e-4a6f-8625-48f32925843e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:22.753003 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e3dea-e91e-4a6f-8625-48f32925843e admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:22.753626 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e3dea-e91e-4a6f-8625-48f32925843e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:22.755634 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e3dea-e91e-4a6f-8625-48f32925843e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:22.782577 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1076/2178] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:22 2022] POST /identity/v3/projects => generated 367 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:22.786823 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b65b8-d833-4d76-9939-bdedcb65a445 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:22.797307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b65b8-d833-4d76-9939-bdedcb65a445 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:22.852653 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133e67f-40fc-4f74-8996-1206d6ae5f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:22.853259 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133e67f-40fc-4f74-8996-1206d6ae5f6a admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:22.853538 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133e67f-40fc-4f74-8996-1206d6ae5f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:22.853808 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133e67f-40fc-4f74-8996-1206d6ae5f6a admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:22.854466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133e67f-40fc-4f74-8996-1206d6ae5f6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:22.856369 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133e67f-40fc-4f74-8996-1206d6ae5f6a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:22.876589 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1103/2179] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:22 2022] POST /identity/v3/users => generated 284 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:22.880651 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e3dea-e91e-4a6f-8625-48f32925843e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:22.891500 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e3dea-e91e-4a6f-8625-48f32925843e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:22.896024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e852ed-aa32-4d39-97d5-a1b89a7db6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:22.896610 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e852ed-aa32-4d39-97d5-a1b89a7db6b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:22.896891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e852ed-aa32-4d39-97d5-a1b89a7db6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:22.897159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e852ed-aa32-4d39-97d5-a1b89a7db6b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:22.897795 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e852ed-aa32-4d39-97d5-a1b89a7db6b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:22.899426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e852ed-aa32-4d39-97d5-a1b89a7db6b1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:22.909854 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1077/2180] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:22.913495 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133e67f-40fc-4f74-8996-1206d6ae5f6a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:22.923567 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133e67f-40fc-4f74-8996-1206d6ae5f6a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:22.928255 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1f3dc-f796-4268-8d46-4b0a6731c2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:22.928830 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1f3dc-f796-4268-8d46-4b0a6731c2ea admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:22.929110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1f3dc-f796-4268-8d46-4b0a6731c2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:22.929383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1f3dc-f796-4268-8d46-4b0a6731c2ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:22.930027 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e1f3dc-f796-4268-8d46-4b0a6731c2ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:22.931650 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e1f3dc-f796-4268-8d46-4b0a6731c2ea admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:22.941981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1104/2181] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:22.946034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e852ed-aa32-4d39-97d5-a1b89a7db6b1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:22.956323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e852ed-aa32-4d39-97d5-a1b89a7db6b1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:22.960987 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a66eb-6451-464a-bf5c-893a5499fd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:22.961645 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a66eb-6451-464a-bf5c-893a5499fd25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:22.961928 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a66eb-6451-464a-bf5c-893a5499fd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:22.962221 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a66eb-6451-464a-bf5c-893a5499fd25 admin admin] PATH_INFO: `/v3/projects/e30146b1079449a1a576b9fa2fde4d5c/users/59be91bfbb4a4c94be1d690065491abc/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:22.977219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a66eb-6451-464a-bf5c-893a5499fd25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e30146b1079449a1a576b9fa2fde4d5c, user_id=59be91bfbb4a4c94be1d690065491abc, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:22.979917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a66eb-6451-464a-bf5c-893a5499fd25 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:22.992542 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1078/2182] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:22 2022] PUT /identity/v3/projects/e30146b1079449a1a576b9fa2fde4d5c/users/59be91bfbb4a4c94be1d690065491abc/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:23.003928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d6236-e905-4f34-85af-afca5eeccddb None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:23.004158 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d6236-e905-4f34-85af-afca5eeccddb None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:23.004437 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d6236-e905-4f34-85af-afca5eeccddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:23.034586 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-818d6236-e905-4f34-85af-afca5eeccddb None None] MFA Rules not processed for user `59be91bfbb4a4c94be1d690065491abc`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:23.036352 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-818d6236-e905-4f34-85af-afca5eeccddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:23.119146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1105/2183] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:23.120457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a66eb-6451-464a-bf5c-893a5499fd25 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:23.129544 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:23.185231 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a66eb-6451-464a-bf5c-893a5499fd25 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:23.189826 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a6049-f8f3-4b1a-9617-bb9e479ce9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:23.190455 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a6049-f8f3-4b1a-9617-bb9e479ce9cd admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:23.190739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a6049-f8f3-4b1a-9617-bb9e479ce9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:23.191014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a6049-f8f3-4b1a-9617-bb9e479ce9cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:23.191650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a6049-f8f3-4b1a-9617-bb9e479ce9cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:23.193544 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a6049-f8f3-4b1a-9617-bb9e479ce9cd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:23.219320 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1079/2184] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:23 2022] POST /identity/v3/projects => generated 367 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:23.224306 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d6236-e905-4f34-85af-afca5eeccddb None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:23.235172 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d6236-e905-4f34-85af-afca5eeccddb None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:23.277204 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:23.293671 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942b169-a7ed-4984-b713-bc7a3010c053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:23.294315 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942b169-a7ed-4984-b713-bc7a3010c053 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:23.294596 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942b169-a7ed-4984-b713-bc7a3010c053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:23.294869 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942b169-a7ed-4984-b713-bc7a3010c053 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:23.295513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942b169-a7ed-4984-b713-bc7a3010c053 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:23.297437 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942b169-a7ed-4984-b713-bc7a3010c053 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:23.318145 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1106/2185] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:23 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:23.323316 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a6049-f8f3-4b1a-9617-bb9e479ce9cd admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:23.333739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a6049-f8f3-4b1a-9617-bb9e479ce9cd admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:23.338349 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6d113-81d5-4760-a063-2f21d5aebf8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:23.338945 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6d113-81d5-4760-a063-2f21d5aebf8a admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:23.339235 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6d113-81d5-4760-a063-2f21d5aebf8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:23.339516 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6d113-81d5-4760-a063-2f21d5aebf8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:23.340155 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6d113-81d5-4760-a063-2f21d5aebf8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:23.341756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6d113-81d5-4760-a063-2f21d5aebf8a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:23.352354 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1080/2186] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:23.356396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942b169-a7ed-4984-b713-bc7a3010c053 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:23.368257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942b169-a7ed-4984-b713-bc7a3010c053 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:23.373356 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05008c2-5647-45ae-9ffc-3551e3814b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:23.374005 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05008c2-5647-45ae-9ffc-3551e3814b87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:23.374323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05008c2-5647-45ae-9ffc-3551e3814b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:23.374599 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05008c2-5647-45ae-9ffc-3551e3814b87 admin admin] PATH_INFO: `/v3/projects/a3f94d353d8941ed9f6b41d105e11653/users/5ee2c288931c48dca86d6f9df677f0c2/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:23.389832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05008c2-5647-45ae-9ffc-3551e3814b87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3f94d353d8941ed9f6b41d105e11653, user_id=5ee2c288931c48dca86d6f9df677f0c2, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:23.392609 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05008c2-5647-45ae-9ffc-3551e3814b87 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:23.405258 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1107/2187] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:23 2022] PUT /identity/v3/projects/a3f94d353d8941ed9f6b41d105e11653/users/5ee2c288931c48dca86d6f9df677f0c2/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:23.410013 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6d113-81d5-4760-a063-2f21d5aebf8a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:23.421208 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6d113-81d5-4760-a063-2f21d5aebf8a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:23.476962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32f858-6a97-482c-bfe9-387627885a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:23.477558 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32f858-6a97-482c-bfe9-387627885a0c admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:23.477838 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32f858-6a97-482c-bfe9-387627885a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:23.478108 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32f858-6a97-482c-bfe9-387627885a0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:23.478790 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32f858-6a97-482c-bfe9-387627885a0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:23.480370 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32f858-6a97-482c-bfe9-387627885a0c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:23.489824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1081/2188] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:23.494716 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05008c2-5647-45ae-9ffc-3551e3814b87 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:23.505068 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05008c2-5647-45ae-9ffc-3551e3814b87 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:23.510266 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dacaa9-9f4a-4898-85cc-d809bcf13f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:23.510911 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dacaa9-9f4a-4898-85cc-d809bcf13f55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:23.511219 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dacaa9-9f4a-4898-85cc-d809bcf13f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:23.511498 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dacaa9-9f4a-4898-85cc-d809bcf13f55 admin admin] PATH_INFO: `/v3/projects/a3f94d353d8941ed9f6b41d105e11653/users/5ee2c288931c48dca86d6f9df677f0c2/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:23.513408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dacaa9-9f4a-4898-85cc-d809bcf13f55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3f94d353d8941ed9f6b41d105e11653, user_id=5ee2c288931c48dca86d6f9df677f0c2, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:23.516132 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dacaa9-9f4a-4898-85cc-d809bcf13f55 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:23.527089 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1108/2189] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:23 2022] PUT /identity/v3/projects/a3f94d353d8941ed9f6b41d105e11653/users/5ee2c288931c48dca86d6f9df677f0c2/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:23.537360 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005233c4-d959-443e-8d8d-49c81fb0c9b1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:23.537609 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005233c4-d959-443e-8d8d-49c81fb0c9b1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:23.537884 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005233c4-d959-443e-8d8d-49c81fb0c9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:23.566595 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-005233c4-d959-443e-8d8d-49c81fb0c9b1 None None] MFA Rules not processed for user `5ee2c288931c48dca86d6f9df677f0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:23.567797 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-005233c4-d959-443e-8d8d-49c81fb0c9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:23.637114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1082/2190] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:23 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:23.648534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dacaa9-9f4a-4898-85cc-d809bcf13f55 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:23.650334 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-62dacaa9-9f4a-4898-85cc-d809bcf13f55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:23.707652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dacaa9-9f4a-4898-85cc-d809bcf13f55 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:23.711210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d20fd9-fba5-4ebb-a386-2af427262d19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:23.711800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d20fd9-fba5-4ebb-a386-2af427262d19 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:23.712077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d20fd9-fba5-4ebb-a386-2af427262d19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:23.712346 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d20fd9-fba5-4ebb-a386-2af427262d19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:23.721375 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d20fd9-fba5-4ebb-a386-2af427262d19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:23.722882 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d20fd9-fba5-4ebb-a386-2af427262d19 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:23.732801 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1109/2191] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:23 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:23.813859 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005233c4-d959-443e-8d8d-49c81fb0c9b1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:23.818454 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:23.824580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005233c4-d959-443e-8d8d-49c81fb0c9b1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:23.828597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5174220-7fb5-492f-bce3-8a493af9b1bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:23.829199 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5174220-7fb5-492f-bce3-8a493af9b1bd service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:23.829483 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5174220-7fb5-492f-bce3-8a493af9b1bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:23.829760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5174220-7fb5-492f-bce3-8a493af9b1bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:23.885008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5174220-7fb5-492f-bce3-8a493af9b1bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:23.886474 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5174220-7fb5-492f-bce3-8a493af9b1bd service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:23.895179 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1083/2192] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:25.455611 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9afc1c-2cf2-4c3f-abac-19440b04b2ea None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:25.456131 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9afc1c-2cf2-4c3f-abac-19440b04b2ea None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:25.456652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9afc1c-2cf2-4c3f-abac-19440b04b2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:25.482443 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2f9afc1c-2cf2-4c3f-abac-19440b04b2ea None None] MFA Rules not processed for user `0d4f8f3a6d42456d8e5cfbe20e54687a`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:25.482443 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2f9afc1c-2cf2-4c3f-abac-19440b04b2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:25.541347 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1110/2193] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:28.692781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5174220-7fb5-492f-bce3-8a493af9b1bd service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:28.757284 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5174220-7fb5-492f-bce3-8a493af9b1bd service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:28.762688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99298b4-c726-4953-9d33-575a65d4146c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:28.763437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99298b4-c726-4953-9d33-575a65d4146c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:28.763721 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99298b4-c726-4953-9d33-575a65d4146c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:28.763990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99298b4-c726-4953-9d33-575a65d4146c admin admin] PATH_INFO: `/v3/users/59be91bfbb4a4c94be1d690065491abc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:28.765151 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99298b4-c726-4953-9d33-575a65d4146c admin admin] RBAC: Authorizing `identity:delete_user(user_id=59be91bfbb4a4c94be1d690065491abc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:28.767099 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99298b4-c726-4953-9d33-575a65d4146c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:28.824374 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b99298b4-c726-4953-9d33-575a65d4146c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59be91bfbb4a4c94be1d690065491abc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:28.831870 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b99298b4-c726-4953-9d33-575a65d4146c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59be91bfbb4a4c94be1d690065491abc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:28.832544 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b99298b4-c726-4953-9d33-575a65d4146c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59be91bfbb4a4c94be1d690065491abc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:28.840396 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b99298b4-c726-4953-9d33-575a65d4146c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59be91bfbb4a4c94be1d690065491abc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:28.856653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b99298b4-c726-4953-9d33-575a65d4146c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59be91bfbb4a4c94be1d690065491abc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:28.863068 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1084/2194] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:33:28 2022] DELETE /identity/v3/users/59be91bfbb4a4c94be1d690065491abc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:28.867730 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcf52f-02ae-4644-a31e-72a18db6ec0c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:28.868200 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcf52f-02ae-4644-a31e-72a18db6ec0c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:28.868453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcf52f-02ae-4644-a31e-72a18db6ec0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:28.890007 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-8bbcf52f-02ae-4644-a31e-72a18db6ec0c None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:28.891254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8bbcf52f-02ae-4644-a31e-72a18db6ec0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:28.933727 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:28.954529 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1111/2195] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:28.964182 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99298b4-c726-4953-9d33-575a65d4146c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:28.965829 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b99298b4-c726-4953-9d33-575a65d4146c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:28.988888 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:29.028117 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99298b4-c726-4953-9d33-575a65d4146c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:29.031881 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0c430-c165-4d92-9120-184ac5dd5888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:29.032564 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0c430-c165-4d92-9120-184ac5dd5888 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:29.032845 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0c430-c165-4d92-9120-184ac5dd5888 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:29.033118 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0c430-c165-4d92-9120-184ac5dd5888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:29.043838 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0c430-c165-4d92-9120-184ac5dd5888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:29.045280 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0c430-c165-4d92-9120-184ac5dd5888 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:29.055866 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1085/2196] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:33:29.342494 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbcf52f-02ae-4644-a31e-72a18db6ec0c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:29.343994 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8bbcf52f-02ae-4644-a31e-72a18db6ec0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:29.358804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbcf52f-02ae-4644-a31e-72a18db6ec0c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:29.364155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e834313-3a59-4e9a-a874-68080e86529f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:29.365374 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e834313-3a59-4e9a-a874-68080e86529f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:29.365665 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e834313-3a59-4e9a-a874-68080e86529f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:29.365942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e834313-3a59-4e9a-a874-68080e86529f admin admin] PATH_INFO: `/v3/projects/e30146b1079449a1a576b9fa2fde4d5c` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:29.367345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e834313-3a59-4e9a-a874-68080e86529f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e30146b1079449a1a576b9fa2fde4d5c)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:29.369327 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e834313-3a59-4e9a-a874-68080e86529f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:29.418115 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7e834313-3a59-4e9a-a874-68080e86529f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e30146b1079449a1a576b9fa2fde4d5c', 'request_id': 'req-7e834313-3a59-4e9a-a874-68080e86529f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:29.424683 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7e834313-3a59-4e9a-a874-68080e86529f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e30146b1079449a1a576b9fa2fde4d5c', 'request_id': 'req-7e834313-3a59-4e9a-a874-68080e86529f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:29.431076 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7e834313-3a59-4e9a-a874-68080e86529f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e30146b1079449a1a576b9fa2fde4d5c', 'request_id': 'req-7e834313-3a59-4e9a-a874-68080e86529f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:29.443251 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7e834313-3a59-4e9a-a874-68080e86529f admin admin] The token cache is being invalidate because project e30146b1079449a1a576b9fa2fde4d5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:33:29.443545 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7e834313-3a59-4e9a-a874-68080e86529f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:29.444177 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7e834313-3a59-4e9a-a874-68080e86529f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:29.445672 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1112/2197] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:33:29 2022] DELETE /identity/v3/projects/e30146b1079449a1a576b9fa2fde4d5c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:29.449724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0c430-c165-4d92-9120-184ac5dd5888 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:29.459250 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0c430-c165-4d92-9120-184ac5dd5888 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:29.514804 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf819d-168b-491e-b22a-a6bce2ce14b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:29.515434 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf819d-168b-491e-b22a-a6bce2ce14b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:29.515710 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf819d-168b-491e-b22a-a6bce2ce14b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:29.515978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf819d-168b-491e-b22a-a6bce2ce14b0 admin admin] PATH_INFO: `/v3/users/5ee2c288931c48dca86d6f9df677f0c2` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:29.517088 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf819d-168b-491e-b22a-a6bce2ce14b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ee2c288931c48dca86d6f9df677f0c2)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:29.519074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf819d-168b-491e-b22a-a6bce2ce14b0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:29.568836 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8bbf819d-168b-491e-b22a-a6bce2ce14b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ee2c288931c48dca86d6f9df677f0c2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:29.576472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8bbf819d-168b-491e-b22a-a6bce2ce14b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ee2c288931c48dca86d6f9df677f0c2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:29.577159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8bbf819d-168b-491e-b22a-a6bce2ce14b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ee2c288931c48dca86d6f9df677f0c2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:29.583954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8bbf819d-168b-491e-b22a-a6bce2ce14b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ee2c288931c48dca86d6f9df677f0c2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:29.596773 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8bbf819d-168b-491e-b22a-a6bce2ce14b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ee2c288931c48dca86d6f9df677f0c2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:29.603251 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1086/2198] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:33:29 2022] DELETE /identity/v3/users/5ee2c288931c48dca86d6f9df677f0c2 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:29.898380 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e834313-3a59-4e9a-a874-68080e86529f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:29.900600 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7e834313-3a59-4e9a-a874-68080e86529f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:29.914028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e834313-3a59-4e9a-a874-68080e86529f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:29.977665 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84490bbc-2a29-4932-ba05-6bff1d14450f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:29.978343 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84490bbc-2a29-4932-ba05-6bff1d14450f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:29.978827 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84490bbc-2a29-4932-ba05-6bff1d14450f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:29.979103 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84490bbc-2a29-4932-ba05-6bff1d14450f admin admin] PATH_INFO: `/v3/projects/a3f94d353d8941ed9f6b41d105e11653` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:29.980235 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84490bbc-2a29-4932-ba05-6bff1d14450f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3f94d353d8941ed9f6b41d105e11653)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:29.982213 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84490bbc-2a29-4932-ba05-6bff1d14450f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:30.030223 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-84490bbc-2a29-4932-ba05-6bff1d14450f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3f94d353d8941ed9f6b41d105e11653', 'request_id': 'req-84490bbc-2a29-4932-ba05-6bff1d14450f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:30.036443 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-84490bbc-2a29-4932-ba05-6bff1d14450f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3f94d353d8941ed9f6b41d105e11653', 'request_id': 'req-84490bbc-2a29-4932-ba05-6bff1d14450f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:30.041920 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-84490bbc-2a29-4932-ba05-6bff1d14450f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3f94d353d8941ed9f6b41d105e11653', 'request_id': 'req-84490bbc-2a29-4932-ba05-6bff1d14450f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:30.052145 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:30.058082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-84490bbc-2a29-4932-ba05-6bff1d14450f admin admin] The token cache is being invalidate because project a3f94d353d8941ed9f6b41d105e11653 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:33:30.058746 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-84490bbc-2a29-4932-ba05-6bff1d14450f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:30.060002 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-84490bbc-2a29-4932-ba05-6bff1d14450f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:30.065899 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1113/2199] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:33:29 2022] DELETE /identity/v3/projects/a3f94d353d8941ed9f6b41d105e11653 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:30.071428 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce4919-d6f9-4001-b4f0-99768903c129 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:30.071942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce4919-d6f9-4001-b4f0-99768903c129 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:30.072046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce4919-d6f9-4001-b4f0-99768903c129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:30.092217 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-14ce4919-d6f9-4001-b4f0-99768903c129 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:30.093403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-14ce4919-d6f9-4001-b4f0-99768903c129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:30.130535 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:30.153064 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1087/2200] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:30.160981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d711163-0fd1-406c-8eee-a3fe382f427f None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:30.161272 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d711163-0fd1-406c-8eee-a3fe382f427f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:30.161552 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d711163-0fd1-406c-8eee-a3fe382f427f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:30.183709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2d711163-0fd1-406c-8eee-a3fe382f427f None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:30.184910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2d711163-0fd1-406c-8eee-a3fe382f427f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:30.192824 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1114/2201] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:30.197542 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce4919-d6f9-4001-b4f0-99768903c129 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:30.207133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce4919-d6f9-4001-b4f0-99768903c129 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:30.211613 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4acda-c33b-45c6-886a-821c94961eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:30.212223 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4acda-c33b-45c6-886a-821c94961eeb admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:30.212538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4acda-c33b-45c6-886a-821c94961eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:30.212813 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4acda-c33b-45c6-886a-821c94961eeb admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:30.213454 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd4acda-c33b-45c6-886a-821c94961eeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:33:30.213757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd4acda-c33b-45c6-886a-821c94961eeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:30.215397 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd4acda-c33b-45c6-886a-821c94961eeb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:30.227063 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1088/2202] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:33:30.231185 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d711163-0fd1-406c-8eee-a3fe382f427f None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:30.240359 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d711163-0fd1-406c-8eee-a3fe382f427f None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:30.245659 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f47a28-50f5-4e4f-ba4c-cde24ea6f4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:30.246290 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f47a28-50f5-4e4f-ba4c-cde24ea6f4e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:30.246569 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f47a28-50f5-4e4f-ba4c-cde24ea6f4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:30.246837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f47a28-50f5-4e4f-ba4c-cde24ea6f4e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:30.247478 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f47a28-50f5-4e4f-ba4c-cde24ea6f4e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:30.249358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f47a28-50f5-4e4f-ba4c-cde24ea6f4e8 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:30.276583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1115/2203] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:30 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:30.280728 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4acda-c33b-45c6-886a-821c94961eeb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:30.291529 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4acda-c33b-45c6-886a-821c94961eeb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:30.348849 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e14d8f-7d1b-4d28-b1d8-7f9d145afd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:30.349521 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e14d8f-7d1b-4d28-b1d8-7f9d145afd7c admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:30.349811 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e14d8f-7d1b-4d28-b1d8-7f9d145afd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:30.350090 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e14d8f-7d1b-4d28-b1d8-7f9d145afd7c admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:30.350759 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e14d8f-7d1b-4d28-b1d8-7f9d145afd7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:30.352681 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e14d8f-7d1b-4d28-b1d8-7f9d145afd7c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:30.373313 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1089/2204] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:30 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:30.377339 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f47a28-50f5-4e4f-ba4c-cde24ea6f4e8 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:30.387989 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f47a28-50f5-4e4f-ba4c-cde24ea6f4e8 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:30.392835 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32ccbd-74bc-44e4-a582-9bb5728df0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:30.393430 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32ccbd-74bc-44e4-a582-9bb5728df0c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:30.393713 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32ccbd-74bc-44e4-a582-9bb5728df0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:30.393988 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32ccbd-74bc-44e4-a582-9bb5728df0c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:30.394678 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32ccbd-74bc-44e4-a582-9bb5728df0c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:30.396269 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32ccbd-74bc-44e4-a582-9bb5728df0c7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:30.407746 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1116/2205] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:30.412036 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e14d8f-7d1b-4d28-b1d8-7f9d145afd7c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:30.422187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e14d8f-7d1b-4d28-b1d8-7f9d145afd7c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:30.426754 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720462dd-4672-4dd4-a5f3-1cf24f7aff86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:30.427355 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720462dd-4672-4dd4-a5f3-1cf24f7aff86 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:30.427675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720462dd-4672-4dd4-a5f3-1cf24f7aff86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:30.427920 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720462dd-4672-4dd4-a5f3-1cf24f7aff86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:30.428727 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720462dd-4672-4dd4-a5f3-1cf24f7aff86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:30.430564 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720462dd-4672-4dd4-a5f3-1cf24f7aff86 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:30.441043 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1090/2206] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:30.445249 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32ccbd-74bc-44e4-a582-9bb5728df0c7 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:30.455810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32ccbd-74bc-44e4-a582-9bb5728df0c7 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:30.460354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e242cd-a317-417a-bd9d-384b01224c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:30.460984 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e242cd-a317-417a-bd9d-384b01224c93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:30.461262 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e242cd-a317-417a-bd9d-384b01224c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:30.461534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e242cd-a317-417a-bd9d-384b01224c93 admin admin] PATH_INFO: `/v3/projects/b30e75b53d8141b9ac4608e3d3720ba9/users/9f6c8abf351441eb98d2bdb8427ab9ee/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:30.476297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e242cd-a317-417a-bd9d-384b01224c93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b30e75b53d8141b9ac4608e3d3720ba9, user_id=9f6c8abf351441eb98d2bdb8427ab9ee, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:30.479026 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e242cd-a317-417a-bd9d-384b01224c93 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:30.491543 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1117/2207] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:30 2022] PUT /identity/v3/projects/b30e75b53d8141b9ac4608e3d3720ba9/users/9f6c8abf351441eb98d2bdb8427ab9ee/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:30.502491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b94656-0b55-45dd-8a90-a8df2ac7f0d6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:30.502780 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b94656-0b55-45dd-8a90-a8df2ac7f0d6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:30.503054 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b94656-0b55-45dd-8a90-a8df2ac7f0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:30.531242 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-33b94656-0b55-45dd-8a90-a8df2ac7f0d6 None None] MFA Rules not processed for user `9f6c8abf351441eb98d2bdb8427ab9ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:30.532438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-33b94656-0b55-45dd-8a90-a8df2ac7f0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:30.598343 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1091/2208] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:30.602814 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e242cd-a317-417a-bd9d-384b01224c93 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:30.663644 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e242cd-a317-417a-bd9d-384b01224c93 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:30.668271 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f04f848-e0d5-48f1-b3a1-c9ff9cccc8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:30.668870 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04f848-e0d5-48f1-b3a1-c9ff9cccc8b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:30.669149 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04f848-e0d5-48f1-b3a1-c9ff9cccc8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:30.669421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04f848-e0d5-48f1-b3a1-c9ff9cccc8b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:30.670066 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f04f848-e0d5-48f1-b3a1-c9ff9cccc8b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:30.671979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f04f848-e0d5-48f1-b3a1-c9ff9cccc8b0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:30.698086 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1118/2209] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:30 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:30.702103 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b94656-0b55-45dd-8a90-a8df2ac7f0d6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:30.713110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b94656-0b55-45dd-8a90-a8df2ac7f0d6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:30.745597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:30.768210 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e27c00b-446a-4a0c-b337-04b56cc38ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:30.768811 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e27c00b-446a-4a0c-b337-04b56cc38ffc admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:30.769092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e27c00b-446a-4a0c-b337-04b56cc38ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:30.769365 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e27c00b-446a-4a0c-b337-04b56cc38ffc admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:30.769990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e27c00b-446a-4a0c-b337-04b56cc38ffc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:30.771948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e27c00b-446a-4a0c-b337-04b56cc38ffc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:30.794437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1092/2210] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:30 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:30.798350 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f04f848-e0d5-48f1-b3a1-c9ff9cccc8b0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:30.808415 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f04f848-e0d5-48f1-b3a1-c9ff9cccc8b0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:30.812899 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a253e09-030c-4915-91a1-499e6df05e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:30.813490 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a253e09-030c-4915-91a1-499e6df05e7b admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:30.813765 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a253e09-030c-4915-91a1-499e6df05e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:30.814035 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a253e09-030c-4915-91a1-499e6df05e7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:30.814709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a253e09-030c-4915-91a1-499e6df05e7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:30.816279 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a253e09-030c-4915-91a1-499e6df05e7b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:30.826551 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1119/2211] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:30.830498 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e27c00b-446a-4a0c-b337-04b56cc38ffc admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:30.840919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e27c00b-446a-4a0c-b337-04b56cc38ffc admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:30.845537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7bd6b7-1c51-426c-b6ca-362efa16fc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:30.846174 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7bd6b7-1c51-426c-b6ca-362efa16fc82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:30.846494 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7bd6b7-1c51-426c-b6ca-362efa16fc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:30.846769 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7bd6b7-1c51-426c-b6ca-362efa16fc82 admin admin] PATH_INFO: `/v3/projects/0b651181e2be433c9dca8f2fe1306872/users/157bc300d38248d794b9212ca1e62026/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:30.861780 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7bd6b7-1c51-426c-b6ca-362efa16fc82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b651181e2be433c9dca8f2fe1306872, user_id=157bc300d38248d794b9212ca1e62026, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:30.864493 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7bd6b7-1c51-426c-b6ca-362efa16fc82 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:30.877121 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1093/2212] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:30 2022] PUT /identity/v3/projects/0b651181e2be433c9dca8f2fe1306872/users/157bc300d38248d794b9212ca1e62026/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:30.880950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a253e09-030c-4915-91a1-499e6df05e7b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:30.891070 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a253e09-030c-4915-91a1-499e6df05e7b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:30.946987 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc8a6d5-4b95-40ab-a8d3-dd134517aed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:30.947692 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8a6d5-4b95-40ab-a8d3-dd134517aed5 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:30.947840 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8a6d5-4b95-40ab-a8d3-dd134517aed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:30.948033 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8a6d5-4b95-40ab-a8d3-dd134517aed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:30.948685 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc8a6d5-4b95-40ab-a8d3-dd134517aed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:30.950286 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc8a6d5-4b95-40ab-a8d3-dd134517aed5 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:30.960134 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1120/2213] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:30.964726 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7bd6b7-1c51-426c-b6ca-362efa16fc82 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:30.974996 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7bd6b7-1c51-426c-b6ca-362efa16fc82 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:30.980267 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce648d1-c405-4c25-824f-a26905f168b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:30.980912 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce648d1-c405-4c25-824f-a26905f168b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:30.981261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce648d1-c405-4c25-824f-a26905f168b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:30.981545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce648d1-c405-4c25-824f-a26905f168b4 admin admin] PATH_INFO: `/v3/projects/0b651181e2be433c9dca8f2fe1306872/users/157bc300d38248d794b9212ca1e62026/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:30.983438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce648d1-c405-4c25-824f-a26905f168b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b651181e2be433c9dca8f2fe1306872, user_id=157bc300d38248d794b9212ca1e62026, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:30.986148 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce648d1-c405-4c25-824f-a26905f168b4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:30.996971 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1094/2214] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:30 2022] PUT /identity/v3/projects/0b651181e2be433c9dca8f2fe1306872/users/157bc300d38248d794b9212ca1e62026/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:31.007433 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bfb09e-c09b-4514-a93d-e0ffd2189bcd None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:31.007726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bfb09e-c09b-4514-a93d-e0ffd2189bcd None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:31.008011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bfb09e-c09b-4514-a93d-e0ffd2189bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:31.015910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:31.049509 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-34bfb09e-c09b-4514-a93d-e0ffd2189bcd None None] MFA Rules not processed for user `157bc300d38248d794b9212ca1e62026`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:31.050795 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-34bfb09e-c09b-4514-a93d-e0ffd2189bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:31.125141 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1121/2215] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:31 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:31.136264 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce648d1-c405-4c25-824f-a26905f168b4 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:31.138044 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cce648d1-c405-4c25-824f-a26905f168b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:31.197646 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce648d1-c405-4c25-824f-a26905f168b4 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:31.201197 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f55fbf-89b6-441f-8828-3d564fa34a5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:31.201790 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f55fbf-89b6-441f-8828-3d564fa34a5e service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:31.202069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f55fbf-89b6-441f-8828-3d564fa34a5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:31.202371 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f55fbf-89b6-441f-8828-3d564fa34a5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:31.211259 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f55fbf-89b6-441f-8828-3d564fa34a5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:31.213386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f55fbf-89b6-441f-8828-3d564fa34a5e service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:31.223083 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1095/2216] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:31 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:31.797033 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bfb09e-c09b-4514-a93d-e0ffd2189bcd None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:31.857906 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bfb09e-c09b-4514-a93d-e0ffd2189bcd None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:31.862446 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f8a84-3888-479e-9a29-545fc0eb50ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:31.863062 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f8a84-3888-479e-9a29-545fc0eb50ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:31.863342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f8a84-3888-479e-9a29-545fc0eb50ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:31.863615 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f8a84-3888-479e-9a29-545fc0eb50ae admin admin] PATH_INFO: `/v3/users/9f6c8abf351441eb98d2bdb8427ab9ee` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:31.864726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f8a84-3888-479e-9a29-545fc0eb50ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f6c8abf351441eb98d2bdb8427ab9ee)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:31.866737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f8a84-3888-479e-9a29-545fc0eb50ae admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:31.912128 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-939f8a84-3888-479e-9a29-545fc0eb50ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f6c8abf351441eb98d2bdb8427ab9ee'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:31.918892 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-939f8a84-3888-479e-9a29-545fc0eb50ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f6c8abf351441eb98d2bdb8427ab9ee'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:31.919543 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-939f8a84-3888-479e-9a29-545fc0eb50ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f6c8abf351441eb98d2bdb8427ab9ee'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:31.920157 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-939f8a84-3888-479e-9a29-545fc0eb50ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f6c8abf351441eb98d2bdb8427ab9ee'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:31.932016 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-939f8a84-3888-479e-9a29-545fc0eb50ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f6c8abf351441eb98d2bdb8427ab9ee'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:31.943658 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1122/2217] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:33:31 2022] DELETE /identity/v3/users/9f6c8abf351441eb98d2bdb8427ab9ee => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:31.948748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585fdac-d04e-456f-9865-e9fce3a6a98a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:31.949051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585fdac-d04e-456f-9865-e9fce3a6a98a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:31.949325 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585fdac-d04e-456f-9865-e9fce3a6a98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:31.968265 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e585fdac-d04e-456f-9865-e9fce3a6a98a None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:31.969381 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e585fdac-d04e-456f-9865-e9fce3a6a98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:32.030657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1096/2218] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:32.040569 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f8a84-3888-479e-9a29-545fc0eb50ae admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:32.042588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-939f8a84-3888-479e-9a29-545fc0eb50ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:32.053538 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:32.104974 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f8a84-3888-479e-9a29-545fc0eb50ae admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:32.108598 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9caef3-3b03-4d86-b28d-08462c7c92e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:32.109200 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9caef3-3b03-4d86-b28d-08462c7c92e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:32.109480 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9caef3-3b03-4d86-b28d-08462c7c92e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:32.109751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9caef3-3b03-4d86-b28d-08462c7c92e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:32.118513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9caef3-3b03-4d86-b28d-08462c7c92e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:32.119934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9caef3-3b03-4d86-b28d-08462c7c92e2 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:32.129150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1123/2219] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:32.438625 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585fdac-d04e-456f-9865-e9fce3a6a98a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:32.440104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e585fdac-d04e-456f-9865-e9fce3a6a98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:32.453060 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585fdac-d04e-456f-9865-e9fce3a6a98a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:32.457634 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1bafb-c8db-4feb-9b95-e518e183d754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:32.458294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1bafb-c8db-4feb-9b95-e518e183d754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:32.458578 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1bafb-c8db-4feb-9b95-e518e183d754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:32.458849 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1bafb-c8db-4feb-9b95-e518e183d754 admin admin] PATH_INFO: `/v3/projects/b30e75b53d8141b9ac4608e3d3720ba9` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:32.460034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d1bafb-c8db-4feb-9b95-e518e183d754 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b30e75b53d8141b9ac4608e3d3720ba9)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:32.461941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d1bafb-c8db-4feb-9b95-e518e183d754 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:32.492927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:32.510789 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d8d1bafb-c8db-4feb-9b95-e518e183d754 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b30e75b53d8141b9ac4608e3d3720ba9', 'request_id': 'req-d8d1bafb-c8db-4feb-9b95-e518e183d754'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:32.516964 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d8d1bafb-c8db-4feb-9b95-e518e183d754 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b30e75b53d8141b9ac4608e3d3720ba9', 'request_id': 'req-d8d1bafb-c8db-4feb-9b95-e518e183d754'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:32.524729 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d8d1bafb-c8db-4feb-9b95-e518e183d754 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b30e75b53d8141b9ac4608e3d3720ba9', 'request_id': 'req-d8d1bafb-c8db-4feb-9b95-e518e183d754'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:32.536188 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d8d1bafb-c8db-4feb-9b95-e518e183d754 admin admin] The token cache is being invalidate because project b30e75b53d8141b9ac4608e3d3720ba9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:33:32.536504 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d8d1bafb-c8db-4feb-9b95-e518e183d754 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:32.537122 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d8d1bafb-c8db-4feb-9b95-e518e183d754 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:32.538663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1097/2220] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:33:32 2022] DELETE /identity/v3/projects/b30e75b53d8141b9ac4608e3d3720ba9 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:32.543150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9caef3-3b03-4d86-b28d-08462c7c92e2 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:32.552880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9caef3-3b03-4d86-b28d-08462c7c92e2 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:32.610439 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27127c9-b1af-4a7d-8e2e-68de5bd02d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:32.611073 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27127c9-b1af-4a7d-8e2e-68de5bd02d9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:32.611356 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27127c9-b1af-4a7d-8e2e-68de5bd02d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:32.611631 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27127c9-b1af-4a7d-8e2e-68de5bd02d9b admin admin] PATH_INFO: `/v3/users/157bc300d38248d794b9212ca1e62026` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:32.612774 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27127c9-b1af-4a7d-8e2e-68de5bd02d9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=157bc300d38248d794b9212ca1e62026)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:32.614736 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27127c9-b1af-4a7d-8e2e-68de5bd02d9b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:32.661214 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f27127c9-b1af-4a7d-8e2e-68de5bd02d9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '157bc300d38248d794b9212ca1e62026'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:32.668522 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f27127c9-b1af-4a7d-8e2e-68de5bd02d9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '157bc300d38248d794b9212ca1e62026'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:32.669183 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f27127c9-b1af-4a7d-8e2e-68de5bd02d9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '157bc300d38248d794b9212ca1e62026'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:32.669794 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f27127c9-b1af-4a7d-8e2e-68de5bd02d9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '157bc300d38248d794b9212ca1e62026'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:32.682063 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f27127c9-b1af-4a7d-8e2e-68de5bd02d9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '157bc300d38248d794b9212ca1e62026'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:32.694693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1124/2221] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:33:32 2022] DELETE /identity/v3/users/157bc300d38248d794b9212ca1e62026 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:32.971011 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1bafb-c8db-4feb-9b95-e518e183d754 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:32.972690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d8d1bafb-c8db-4feb-9b95-e518e183d754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:33.001372 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1bafb-c8db-4feb-9b95-e518e183d754 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:33.067322 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e1c42-890f-43ab-ba5e-d7f560296cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:33.067982 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e1c42-890f-43ab-ba5e-d7f560296cef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:33.068337 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e1c42-890f-43ab-ba5e-d7f560296cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:33.068616 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e1c42-890f-43ab-ba5e-d7f560296cef admin admin] PATH_INFO: `/v3/projects/0b651181e2be433c9dca8f2fe1306872` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:33.069755 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7e1c42-890f-43ab-ba5e-d7f560296cef admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b651181e2be433c9dca8f2fe1306872)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:33.072415 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7e1c42-890f-43ab-ba5e-d7f560296cef admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:33.119399 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0d7e1c42-890f-43ab-ba5e-d7f560296cef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b651181e2be433c9dca8f2fe1306872', 'request_id': 'req-0d7e1c42-890f-43ab-ba5e-d7f560296cef'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:33.125635 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0d7e1c42-890f-43ab-ba5e-d7f560296cef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b651181e2be433c9dca8f2fe1306872', 'request_id': 'req-0d7e1c42-890f-43ab-ba5e-d7f560296cef'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:33.134840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0d7e1c42-890f-43ab-ba5e-d7f560296cef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b651181e2be433c9dca8f2fe1306872', 'request_id': 'req-0d7e1c42-890f-43ab-ba5e-d7f560296cef'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:33.146047 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0d7e1c42-890f-43ab-ba5e-d7f560296cef admin admin] The token cache is being invalidate because project 0b651181e2be433c9dca8f2fe1306872 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:33:33.146492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0d7e1c42-890f-43ab-ba5e-d7f560296cef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:33.147129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0d7e1c42-890f-43ab-ba5e-d7f560296cef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:33.148618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1098/2222] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:33:32 2022] DELETE /identity/v3/projects/0b651181e2be433c9dca8f2fe1306872 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:33.153500 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859647b-0719-4cf3-90ce-f8ddad5480d8 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:33.153981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859647b-0719-4cf3-90ce-f8ddad5480d8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:33.154175 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859647b-0719-4cf3-90ce-f8ddad5480d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:33.173909 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7859647b-0719-4cf3-90ce-f8ddad5480d8 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:33.175150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7859647b-0719-4cf3-90ce-f8ddad5480d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:33.203111 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:33.239860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1125/2223] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:33.248242 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac154495-79a4-47d3-b942-64cc35bc0ff6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:33.248614 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac154495-79a4-47d3-b942-64cc35bc0ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:33.248962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac154495-79a4-47d3-b942-64cc35bc0ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:33.269766 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ac154495-79a4-47d3-b942-64cc35bc0ff6 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:33.271014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ac154495-79a4-47d3-b942-64cc35bc0ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:33.279066 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1099/2224] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:33.283564 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7859647b-0719-4cf3-90ce-f8ddad5480d8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:33.293927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7859647b-0719-4cf3-90ce-f8ddad5480d8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:33.298561 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd1c2a-40a0-4569-b536-51c9c5eed4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:33.299156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd1c2a-40a0-4569-b536-51c9c5eed4b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:33.299495 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd1c2a-40a0-4569-b536-51c9c5eed4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:33.299705 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd1c2a-40a0-4569-b536-51c9c5eed4b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:33.300351 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cd1c2a-40a0-4569-b536-51c9c5eed4b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:33:33.300653 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cd1c2a-40a0-4569-b536-51c9c5eed4b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:33.302246 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cd1c2a-40a0-4569-b536-51c9c5eed4b1 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:33.317550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1126/2225] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:33:33.318774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac154495-79a4-47d3-b942-64cc35bc0ff6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:33.328164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac154495-79a4-47d3-b942-64cc35bc0ff6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:33.332743 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf361587-7175-4fc4-9c36-5e99ea293390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:33.333335 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf361587-7175-4fc4-9c36-5e99ea293390 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:33.333615 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf361587-7175-4fc4-9c36-5e99ea293390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:33.333885 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf361587-7175-4fc4-9c36-5e99ea293390 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:33.334617 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf361587-7175-4fc4-9c36-5e99ea293390 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:33.336799 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf361587-7175-4fc4-9c36-5e99ea293390 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:33.364246 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1100/2226] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:33 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:33.368962 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd1c2a-40a0-4569-b536-51c9c5eed4b1 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:33.379400 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd1c2a-40a0-4569-b536-51c9c5eed4b1 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:33.436278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08547c47-7adb-4e71-9e76-b18872c3b4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:33.436883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08547c47-7adb-4e71-9e76-b18872c3b4b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:33.437157 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08547c47-7adb-4e71-9e76-b18872c3b4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:33.437424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08547c47-7adb-4e71-9e76-b18872c3b4b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:33.438072 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08547c47-7adb-4e71-9e76-b18872c3b4b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:33.439999 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08547c47-7adb-4e71-9e76-b18872c3b4b1 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:33.462231 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1127/2227] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:33 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:33.465921 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf361587-7175-4fc4-9c36-5e99ea293390 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:33.476197 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf361587-7175-4fc4-9c36-5e99ea293390 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:33.481398 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40d1c6-3178-4f0c-9f08-f8d11af21282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:33.482003 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40d1c6-3178-4f0c-9f08-f8d11af21282 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:33.482314 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40d1c6-3178-4f0c-9f08-f8d11af21282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:33.482592 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40d1c6-3178-4f0c-9f08-f8d11af21282 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:33.483238 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff40d1c6-3178-4f0c-9f08-f8d11af21282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:33.484821 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff40d1c6-3178-4f0c-9f08-f8d11af21282 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:33.495631 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1101/2228] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:33.499588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08547c47-7adb-4e71-9e76-b18872c3b4b1 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:33.509637 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08547c47-7adb-4e71-9e76-b18872c3b4b1 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:33.514815 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404f2d0-4b8f-4a01-948e-7b427adc5d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:33.515428 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404f2d0-4b8f-4a01-948e-7b427adc5d35 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:33.515856 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404f2d0-4b8f-4a01-948e-7b427adc5d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:33.516169 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404f2d0-4b8f-4a01-948e-7b427adc5d35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:33.516825 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404f2d0-4b8f-4a01-948e-7b427adc5d35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:33.518456 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404f2d0-4b8f-4a01-948e-7b427adc5d35 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:33.530088 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1128/2229] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:33.534699 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40d1c6-3178-4f0c-9f08-f8d11af21282 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:33.545225 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40d1c6-3178-4f0c-9f08-f8d11af21282 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:33.549898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f59bd6-1bef-4ce3-84b1-8709de43ed71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:33.550516 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f59bd6-1bef-4ce3-84b1-8709de43ed71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:33.550802 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f59bd6-1bef-4ce3-84b1-8709de43ed71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:33.551080 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f59bd6-1bef-4ce3-84b1-8709de43ed71 admin admin] PATH_INFO: `/v3/projects/5533c2f2ac4841ceaa5b30d73e3be569/users/57b6bdcf05234b1596ebe24afb337d6b/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:33.553299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:33.568713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f59bd6-1bef-4ce3-84b1-8709de43ed71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5533c2f2ac4841ceaa5b30d73e3be569, user_id=57b6bdcf05234b1596ebe24afb337d6b, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:33.571613 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f59bd6-1bef-4ce3-84b1-8709de43ed71 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:33.585297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1102/2230] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:33 2022] PUT /identity/v3/projects/5533c2f2ac4841ceaa5b30d73e3be569/users/57b6bdcf05234b1596ebe24afb337d6b/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:33:33.596409 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb4abb-f596-4fc7-9a69-fee5907acccb None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:33.596696 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb4abb-f596-4fc7-9a69-fee5907acccb None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:33.596973 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb4abb-f596-4fc7-9a69-fee5907acccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:33.625440 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-01cb4abb-f596-4fc7-9a69-fee5907acccb None None] MFA Rules not processed for user `57b6bdcf05234b1596ebe24afb337d6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:33.626689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-01cb4abb-f596-4fc7-9a69-fee5907acccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:33.689738 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1129/2231] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:33.700808 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f59bd6-1bef-4ce3-84b1-8709de43ed71 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:33.702504 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-86f59bd6-1bef-4ce3-84b1-8709de43ed71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:33.759994 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f59bd6-1bef-4ce3-84b1-8709de43ed71 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:33.763794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841d79b-4e4f-48bf-af35-e2764e02efd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:33.764434 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841d79b-4e4f-48bf-af35-e2764e02efd1 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:33.764743 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841d79b-4e4f-48bf-af35-e2764e02efd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:33.765016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841d79b-4e4f-48bf-af35-e2764e02efd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:33.773428 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841d79b-4e4f-48bf-af35-e2764e02efd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:33.775397 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841d79b-4e4f-48bf-af35-e2764e02efd1 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:33.784590 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1103/2232] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:33.806533 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb4abb-f596-4fc7-9a69-fee5907acccb None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:33.835565 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:33.871769 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb4abb-f596-4fc7-9a69-fee5907acccb None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:33.876350 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614f04d-b6a9-4b10-98a2-36442adfb40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:33.876979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614f04d-b6a9-4b10-98a2-36442adfb40e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:33.877261 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614f04d-b6a9-4b10-98a2-36442adfb40e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:33.877537 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614f04d-b6a9-4b10-98a2-36442adfb40e admin admin] PATH_INFO: `/v3/users/57b6bdcf05234b1596ebe24afb337d6b` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:33.878708 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3614f04d-b6a9-4b10-98a2-36442adfb40e admin admin] RBAC: Authorizing `identity:delete_user(user_id=57b6bdcf05234b1596ebe24afb337d6b)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:33.880686 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3614f04d-b6a9-4b10-98a2-36442adfb40e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:33.926809 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3614f04d-b6a9-4b10-98a2-36442adfb40e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57b6bdcf05234b1596ebe24afb337d6b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:33.934009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3614f04d-b6a9-4b10-98a2-36442adfb40e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57b6bdcf05234b1596ebe24afb337d6b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:33.934724 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3614f04d-b6a9-4b10-98a2-36442adfb40e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57b6bdcf05234b1596ebe24afb337d6b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:33.935349 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3614f04d-b6a9-4b10-98a2-36442adfb40e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57b6bdcf05234b1596ebe24afb337d6b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:33.947167 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3614f04d-b6a9-4b10-98a2-36442adfb40e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57b6bdcf05234b1596ebe24afb337d6b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:33.959778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1130/2233] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:33:33 2022] DELETE /identity/v3/users/57b6bdcf05234b1596ebe24afb337d6b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:33.964799 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838dcbb-1b87-459b-ac78-fc2de29a78c0 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:33.965087 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838dcbb-1b87-459b-ac78-fc2de29a78c0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:33.965361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838dcbb-1b87-459b-ac78-fc2de29a78c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:33.984612 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c838dcbb-1b87-459b-ac78-fc2de29a78c0 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:33.985822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c838dcbb-1b87-459b-ac78-fc2de29a78c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:34.044604 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1104/2234] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:34.055857 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614f04d-b6a9-4b10-98a2-36442adfb40e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:34.057309 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3614f04d-b6a9-4b10-98a2-36442adfb40e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:34.117349 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614f04d-b6a9-4b10-98a2-36442adfb40e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:34.121057 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100b50e-e757-4a82-ac86-1b916cb0e740 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:34.121651 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100b50e-e757-4a82-ac86-1b916cb0e740 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:34.121927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100b50e-e757-4a82-ac86-1b916cb0e740 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:34.122216 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100b50e-e757-4a82-ac86-1b916cb0e740 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:34.131472 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6100b50e-e757-4a82-ac86-1b916cb0e740 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:34.132897 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6100b50e-e757-4a82-ac86-1b916cb0e740 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:34.142916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1131/2235] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:33:34.419149 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838dcbb-1b87-459b-ac78-fc2de29a78c0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:34.420594 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c838dcbb-1b87-459b-ac78-fc2de29a78c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:34.435964 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838dcbb-1b87-459b-ac78-fc2de29a78c0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:34.441941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08011fa-59d6-47fb-ab3d-a5b46436278b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:34.442641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08011fa-59d6-47fb-ab3d-a5b46436278b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:34.442920 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08011fa-59d6-47fb-ab3d-a5b46436278b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:34.443190 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08011fa-59d6-47fb-ab3d-a5b46436278b admin admin] PATH_INFO: `/v3/projects/5533c2f2ac4841ceaa5b30d73e3be569` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:34.444314 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08011fa-59d6-47fb-ab3d-a5b46436278b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5533c2f2ac4841ceaa5b30d73e3be569)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:34.446247 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08011fa-59d6-47fb-ab3d-a5b46436278b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:34.493706 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c08011fa-59d6-47fb-ab3d-a5b46436278b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5533c2f2ac4841ceaa5b30d73e3be569', 'request_id': 'req-c08011fa-59d6-47fb-ab3d-a5b46436278b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:34.499925 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c08011fa-59d6-47fb-ab3d-a5b46436278b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5533c2f2ac4841ceaa5b30d73e3be569', 'request_id': 'req-c08011fa-59d6-47fb-ab3d-a5b46436278b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:34.507348 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c08011fa-59d6-47fb-ab3d-a5b46436278b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5533c2f2ac4841ceaa5b30d73e3be569', 'request_id': 'req-c08011fa-59d6-47fb-ab3d-a5b46436278b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:34.518492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c08011fa-59d6-47fb-ab3d-a5b46436278b admin admin] The token cache is being invalidate because project 5533c2f2ac4841ceaa5b30d73e3be569 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:33:34.518789 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c08011fa-59d6-47fb-ab3d-a5b46436278b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:34.519408 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c08011fa-59d6-47fb-ab3d-a5b46436278b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:34.520900 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1105/2236] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:33:34 2022] DELETE /identity/v3/projects/5533c2f2ac4841ceaa5b30d73e3be569 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:34.525933 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49fa65-24dd-48d9-b926-4ebd38707784 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:34.526403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49fa65-24dd-48d9-b926-4ebd38707784 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:34.526552 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49fa65-24dd-48d9-b926-4ebd38707784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:34.546481 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-4b49fa65-24dd-48d9-b926-4ebd38707784 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:34.547692 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4b49fa65-24dd-48d9-b926-4ebd38707784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:34.605621 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:34.639778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1132/2237] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:34.647448 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3b9f7-8f8e-4e86-9593-0b462dc8a45e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:34.647758 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3b9f7-8f8e-4e86-9593-0b462dc8a45e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:34.648040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3b9f7-8f8e-4e86-9593-0b462dc8a45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:34.668887 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e2b3b9f7-8f8e-4e86-9593-0b462dc8a45e None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:34.670102 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e2b3b9f7-8f8e-4e86-9593-0b462dc8a45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:34.679702 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1106/2238] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:34.684532 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49fa65-24dd-48d9-b926-4ebd38707784 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:34.694852 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49fa65-24dd-48d9-b926-4ebd38707784 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:34.701443 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d29c7-74d1-4107-ac41-3c313334fbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:34.702220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d29c7-74d1-4107-ac41-3c313334fbcc admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:34.702606 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d29c7-74d1-4107-ac41-3c313334fbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:34.703002 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d29c7-74d1-4107-ac41-3c313334fbcc admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:34.703766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d29c7-74d1-4107-ac41-3c313334fbcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:33:34.704296 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d29c7-74d1-4107-ac41-3c313334fbcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:34.706004 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d29c7-74d1-4107-ac41-3c313334fbcc admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:34.719698 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1133/2239] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:33:34.724291 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3b9f7-8f8e-4e86-9593-0b462dc8a45e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:34.729658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:34.740375 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3b9f7-8f8e-4e86-9593-0b462dc8a45e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:34.745341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c8869-c841-479c-bbc9-aaaf49b170c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:34.746019 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c8869-c841-479c-bbc9-aaaf49b170c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:34.746397 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c8869-c841-479c-bbc9-aaaf49b170c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:34.746758 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c8869-c841-479c-bbc9-aaaf49b170c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:34.747469 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c8869-c841-479c-bbc9-aaaf49b170c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:34.749434 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c8869-c841-479c-bbc9-aaaf49b170c9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:34.780872 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1107/2240] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:34 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:34.782073 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d29c7-74d1-4107-ac41-3c313334fbcc admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:34.793402 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d29c7-74d1-4107-ac41-3c313334fbcc admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:34.851882 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065b406-a544-46f2-8e3d-270233b200e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:34.852533 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065b406-a544-46f2-8e3d-270233b200e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:34.852819 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065b406-a544-46f2-8e3d-270233b200e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:34.853090 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065b406-a544-46f2-8e3d-270233b200e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:34.853729 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065b406-a544-46f2-8e3d-270233b200e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:34.855735 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065b406-a544-46f2-8e3d-270233b200e6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:34.876444 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1134/2241] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:34 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:33:34.880489 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c8869-c841-479c-bbc9-aaaf49b170c9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:34.891298 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c8869-c841-479c-bbc9-aaaf49b170c9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:34.895947 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cad027-b7d0-4458-8033-7110227a9f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:34.896549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cad027-b7d0-4458-8033-7110227a9f44 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:34.896831 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cad027-b7d0-4458-8033-7110227a9f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:34.897106 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cad027-b7d0-4458-8033-7110227a9f44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:34.897749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cad027-b7d0-4458-8033-7110227a9f44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:34.899468 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cad027-b7d0-4458-8033-7110227a9f44 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:34.909979 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1108/2242] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:34.914018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065b406-a544-46f2-8e3d-270233b200e6 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:34.923952 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065b406-a544-46f2-8e3d-270233b200e6 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:34.928816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcf613-8901-4e7f-ae60-5c4999513a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:34.929461 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcf613-8901-4e7f-ae60-5c4999513a35 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:34.929742 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcf613-8901-4e7f-ae60-5c4999513a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:34.930012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcf613-8901-4e7f-ae60-5c4999513a35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:34.930718 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedcf613-8901-4e7f-ae60-5c4999513a35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:34.932359 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedcf613-8901-4e7f-ae60-5c4999513a35 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:34.942873 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1135/2243] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:34.947307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cad027-b7d0-4458-8033-7110227a9f44 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:34.957362 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cad027-b7d0-4458-8033-7110227a9f44 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:34.962598 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd80d77-0323-42f9-a957-541fae32a832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:34.963246 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd80d77-0323-42f9-a957-541fae32a832 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:34.963532 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd80d77-0323-42f9-a957-541fae32a832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:34.963812 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd80d77-0323-42f9-a957-541fae32a832 admin admin] PATH_INFO: `/v3/projects/779d894748c0492ca2a9978f0cfde0f5/users/8dd9deced4124568b11a6512c5489753/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:34.978301 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd80d77-0323-42f9-a957-541fae32a832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=779d894748c0492ca2a9978f0cfde0f5, user_id=8dd9deced4124568b11a6512c5489753, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:34.980953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd80d77-0323-42f9-a957-541fae32a832 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:34.993420 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1109/2244] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:34 2022] PUT /identity/v3/projects/779d894748c0492ca2a9978f0cfde0f5/users/8dd9deced4124568b11a6512c5489753/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:35.004330 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9236bbf-98f8-42bc-98ee-be4fadd179d4 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:35.004636 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9236bbf-98f8-42bc-98ee-be4fadd179d4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:35.004935 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9236bbf-98f8-42bc-98ee-be4fadd179d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:35.033425 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e9236bbf-98f8-42bc-98ee-be4fadd179d4 None None] MFA Rules not processed for user `8dd9deced4124568b11a6512c5489753`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:35.034695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e9236bbf-98f8-42bc-98ee-be4fadd179d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:35.113328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1136/2245] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:33:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:35.125744 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd80d77-0323-42f9-a957-541fae32a832 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:35.127750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ecd80d77-0323-42f9-a957-541fae32a832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:35.217116 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd80d77-0323-42f9-a957-541fae32a832 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:35.220939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d584c-7b2c-4748-a28b-fe6ceef4ad05 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2c872129414105a112661cc2b477e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:35.221572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d584c-7b2c-4748-a28b-fe6ceef4ad05 service glance] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:35.221883 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d584c-7b2c-4748-a28b-fe6ceef4ad05 service glance] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:35.222121 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d584c-7b2c-4748-a28b-fe6ceef4ad05 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:35.231475 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d584c-7b2c-4748-a28b-fe6ceef4ad05 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:35.233140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d584c-7b2c-4748-a28b-fe6ceef4ad05 service glance] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:35.243242 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1110/2246] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:45.110303 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=77,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:45.238911 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=58,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:05.494389 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9236bbf-98f8-42bc-98ee-be4fadd179d4 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:05.555667 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9236bbf-98f8-42bc-98ee-be4fadd179d4 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:05.560111 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f49dc-f112-4d04-a342-e2393cce4195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:05.561353 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f49dc-f112-4d04-a342-e2393cce4195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:05.561633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f49dc-f112-4d04-a342-e2393cce4195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:05.561905 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f49dc-f112-4d04-a342-e2393cce4195 admin admin] PATH_INFO: `/v3/users/8dd9deced4124568b11a6512c5489753` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:05.563066 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f49dc-f112-4d04-a342-e2393cce4195 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dd9deced4124568b11a6512c5489753)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:05.564957 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f49dc-f112-4d04-a342-e2393cce4195 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:05.609498 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bd3f49dc-f112-4d04-a342-e2393cce4195 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dd9deced4124568b11a6512c5489753'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:05.616301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bd3f49dc-f112-4d04-a342-e2393cce4195 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dd9deced4124568b11a6512c5489753'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:05.616950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bd3f49dc-f112-4d04-a342-e2393cce4195 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dd9deced4124568b11a6512c5489753'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:05.617547 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bd3f49dc-f112-4d04-a342-e2393cce4195 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dd9deced4124568b11a6512c5489753'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:05.630042 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bd3f49dc-f112-4d04-a342-e2393cce4195 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dd9deced4124568b11a6512c5489753'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:05.642449 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1137/2247] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:05 2022] DELETE /identity/v3/users/8dd9deced4124568b11a6512c5489753 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:05.646856 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d42ba1-6756-4fa3-9312-dc9faa8978f3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:05.647359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d42ba1-6756-4fa3-9312-dc9faa8978f3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:05.647468 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d42ba1-6756-4fa3-9312-dc9faa8978f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:05.667876 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-31d42ba1-6756-4fa3-9312-dc9faa8978f3 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:05.669225 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-31d42ba1-6756-4fa3-9312-dc9faa8978f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:05.730277 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1111/2248] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:34:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:05.740098 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f49dc-f112-4d04-a342-e2393cce4195 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:05.741503 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bd3f49dc-f112-4d04-a342-e2393cce4195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:05.797770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f49dc-f112-4d04-a342-e2393cce4195 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:05.801706 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21b58c-f05e-4d5c-b9b6-8df8ec1aa01b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:05.802337 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21b58c-f05e-4d5c-b9b6-8df8ec1aa01b service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:05.802622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21b58c-f05e-4d5c-b9b6-8df8ec1aa01b service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:05.802896 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21b58c-f05e-4d5c-b9b6-8df8ec1aa01b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:05.812527 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b21b58c-f05e-4d5c-b9b6-8df8ec1aa01b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:05.813965 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b21b58c-f05e-4d5c-b9b6-8df8ec1aa01b service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:05.823227 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1138/2249] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:06.092944 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d42ba1-6756-4fa3-9312-dc9faa8978f3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:06.094429 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-31d42ba1-6756-4fa3-9312-dc9faa8978f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:06.106401 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d42ba1-6756-4fa3-9312-dc9faa8978f3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:06.111704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4fbeb-d275-4d66-ad4e-7e0aacf2a743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:06.112339 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4fbeb-d275-4d66-ad4e-7e0aacf2a743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:06.112618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4fbeb-d275-4d66-ad4e-7e0aacf2a743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:06.112889 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4fbeb-d275-4d66-ad4e-7e0aacf2a743 admin admin] PATH_INFO: `/v3/projects/779d894748c0492ca2a9978f0cfde0f5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:06.114020 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4fbeb-d275-4d66-ad4e-7e0aacf2a743 admin admin] RBAC: Authorizing `identity:delete_project(project_id=779d894748c0492ca2a9978f0cfde0f5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:06.115969 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4fbeb-d275-4d66-ad4e-7e0aacf2a743 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:06.165714 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4ec4fbeb-d275-4d66-ad4e-7e0aacf2a743 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '779d894748c0492ca2a9978f0cfde0f5', 'request_id': 'req-4ec4fbeb-d275-4d66-ad4e-7e0aacf2a743'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:06.172531 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4ec4fbeb-d275-4d66-ad4e-7e0aacf2a743 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '779d894748c0492ca2a9978f0cfde0f5', 'request_id': 'req-4ec4fbeb-d275-4d66-ad4e-7e0aacf2a743'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:06.179821 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4ec4fbeb-d275-4d66-ad4e-7e0aacf2a743 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '779d894748c0492ca2a9978f0cfde0f5', 'request_id': 'req-4ec4fbeb-d275-4d66-ad4e-7e0aacf2a743'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:06.190856 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4ec4fbeb-d275-4d66-ad4e-7e0aacf2a743 admin admin] The token cache is being invalidate because project 779d894748c0492ca2a9978f0cfde0f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:06.191142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4ec4fbeb-d275-4d66-ad4e-7e0aacf2a743 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:06.191758 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4ec4fbeb-d275-4d66-ad4e-7e0aacf2a743 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:06.193226 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1112/2250] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:06 2022] DELETE /identity/v3/projects/779d894748c0492ca2a9978f0cfde0f5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:06.198222 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d03b7-8440-4de4-ac2c-ad07ccc24934 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:06.198519 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d03b7-8440-4de4-ac2c-ad07ccc24934 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:06.198813 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d03b7-8440-4de4-ac2c-ad07ccc24934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:06.217695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ea8d03b7-8440-4de4-ac2c-ad07ccc24934 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:06.218909 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ea8d03b7-8440-4de4-ac2c-ad07ccc24934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:06.236202 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:06.280517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1139/2251] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:34:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:06.289824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f644b-3dba-4e4e-a026-2a2d19ea8fa5 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:06.290112 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f644b-3dba-4e4e-a026-2a2d19ea8fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:06.290445 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f644b-3dba-4e4e-a026-2a2d19ea8fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:06.314609 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fc3f644b-3dba-4e4e-a026-2a2d19ea8fa5 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:06.315886 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fc3f644b-3dba-4e4e-a026-2a2d19ea8fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:06.323741 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1113/2252] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:34:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:06.327939 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d03b7-8440-4de4-ac2c-ad07ccc24934 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:06.337471 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d03b7-8440-4de4-ac2c-ad07ccc24934 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:06.341986 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463acaa6-aed3-4c09-ab5e-614775e11ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:06.342604 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463acaa6-aed3-4c09-ab5e-614775e11ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:06.342880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463acaa6-aed3-4c09-ab5e-614775e11ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:06.343146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463acaa6-aed3-4c09-ab5e-614775e11ea5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:06.343787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463acaa6-aed3-4c09-ab5e-614775e11ea5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:34:06.344089 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463acaa6-aed3-4c09-ab5e-614775e11ea5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:06.345657 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463acaa6-aed3-4c09-ab5e-614775e11ea5 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:06.356755 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1140/2253] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:34:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:34:06.361274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f644b-3dba-4e4e-a026-2a2d19ea8fa5 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:06.370508 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f644b-3dba-4e4e-a026-2a2d19ea8fa5 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:06.375769 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7b945-71df-4fdf-af58-3fcf9a5e7c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:06.376359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7b945-71df-4fdf-af58-3fcf9a5e7c9f admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:06.376654 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7b945-71df-4fdf-af58-3fcf9a5e7c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:06.376921 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7b945-71df-4fdf-af58-3fcf9a5e7c9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:06.377551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f7b945-71df-4fdf-af58-3fcf9a5e7c9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:06.379474 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f7b945-71df-4fdf-af58-3fcf9a5e7c9f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:06.408812 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1114/2254] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:06 2022] POST /identity/v3/projects => generated 361 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:06.413408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463acaa6-aed3-4c09-ab5e-614775e11ea5 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:06.423732 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463acaa6-aed3-4c09-ab5e-614775e11ea5 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:06.480065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b485297-381b-48d8-97d2-84c8b42498a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:06.480669 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b485297-381b-48d8-97d2-84c8b42498a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:06.480949 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b485297-381b-48d8-97d2-84c8b42498a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:06.481222 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b485297-381b-48d8-97d2-84c8b42498a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:06.481868 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b485297-381b-48d8-97d2-84c8b42498a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:06.484319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b485297-381b-48d8-97d2-84c8b42498a0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:06.505659 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1141/2255] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:06 2022] POST /identity/v3/users => generated 281 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:06.509706 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7b945-71df-4fdf-af58-3fcf9a5e7c9f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:06.520056 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7b945-71df-4fdf-af58-3fcf9a5e7c9f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:06.524610 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951c07a-3f06-4527-9cd8-0783019e6ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:06.525200 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951c07a-3f06-4527-9cd8-0783019e6ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:06.525486 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951c07a-3f06-4527-9cd8-0783019e6ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:06.525760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951c07a-3f06-4527-9cd8-0783019e6ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:06.526426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951c07a-3f06-4527-9cd8-0783019e6ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:06.528015 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951c07a-3f06-4527-9cd8-0783019e6ea7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:06.538532 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1115/2256] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:06.542274 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b485297-381b-48d8-97d2-84c8b42498a0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:06.552553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b485297-381b-48d8-97d2-84c8b42498a0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:06.557028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12928c1-628f-450e-901c-a3d40241f514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:06.557610 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12928c1-628f-450e-901c-a3d40241f514 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:06.557885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12928c1-628f-450e-901c-a3d40241f514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:06.558156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12928c1-628f-450e-901c-a3d40241f514 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:06.558826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12928c1-628f-450e-901c-a3d40241f514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:06.560981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12928c1-628f-450e-901c-a3d40241f514 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:06.572065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1142/2257] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:06.577258 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951c07a-3f06-4527-9cd8-0783019e6ea7 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:06.588864 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951c07a-3f06-4527-9cd8-0783019e6ea7 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:06.595701 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daab89c-f9c1-4099-bce5-8c9c0f8aa658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:06.596419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daab89c-f9c1-4099-bce5-8c9c0f8aa658 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:06.596704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daab89c-f9c1-4099-bce5-8c9c0f8aa658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:06.596980 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daab89c-f9c1-4099-bce5-8c9c0f8aa658 admin admin] PATH_INFO: `/v3/projects/77395891bff3497e9a8e063a4e3be13d/users/f2fe5d539cbf4153a2dcbe7b28509fc6/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:06.614885 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daab89c-f9c1-4099-bce5-8c9c0f8aa658 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77395891bff3497e9a8e063a4e3be13d, user_id=f2fe5d539cbf4153a2dcbe7b28509fc6, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:06.617534 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daab89c-f9c1-4099-bce5-8c9c0f8aa658 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:06.630586 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1116/2258] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:06 2022] PUT /identity/v3/projects/77395891bff3497e9a8e063a4e3be13d/users/f2fe5d539cbf4153a2dcbe7b28509fc6/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:06.640719 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91b79f-02d4-4a19-8897-669da7bcbe38 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:06.641199 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91b79f-02d4-4a19-8897-669da7bcbe38 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:06.641472 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91b79f-02d4-4a19-8897-669da7bcbe38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:06.670957 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7b91b79f-02d4-4a19-8897-669da7bcbe38 None None] MFA Rules not processed for user `f2fe5d539cbf4153a2dcbe7b28509fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:06.672167 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7b91b79f-02d4-4a19-8897-669da7bcbe38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:06.735909 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1143/2259] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:34:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:34:06.768553 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daab89c-f9c1-4099-bce5-8c9c0f8aa658 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:06.770777 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9daab89c-f9c1-4099-bce5-8c9c0f8aa658 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:06.793206 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:06.831618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daab89c-f9c1-4099-bce5-8c9c0f8aa658 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:06.835249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0be1b2-3cca-4dde-b484-ed97e22ad8fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:06.835847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0be1b2-3cca-4dde-b484-ed97e22ad8fa service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:06.836132 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0be1b2-3cca-4dde-b484-ed97e22ad8fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:06.836407 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0be1b2-3cca-4dde-b484-ed97e22ad8fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:06.845132 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0be1b2-3cca-4dde-b484-ed97e22ad8fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:06.846595 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0be1b2-3cca-4dde-b484-ed97e22ad8fa service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:06.855366 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1117/2260] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:06.871674 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91b79f-02d4-4a19-8897-669da7bcbe38 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:06.890713 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:06.934526 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91b79f-02d4-4a19-8897-669da7bcbe38 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:06.939070 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e9dad-c979-43d7-87a5-217dd18e2b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:06.939687 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e9dad-c979-43d7-87a5-217dd18e2b4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:06.939965 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e9dad-c979-43d7-87a5-217dd18e2b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:06.940235 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e9dad-c979-43d7-87a5-217dd18e2b4d admin admin] PATH_INFO: `/v3/users/f2fe5d539cbf4153a2dcbe7b28509fc6` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:06.941382 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e9dad-c979-43d7-87a5-217dd18e2b4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2fe5d539cbf4153a2dcbe7b28509fc6)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:06.943327 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e9dad-c979-43d7-87a5-217dd18e2b4d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:06.989877 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-471e9dad-c979-43d7-87a5-217dd18e2b4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2fe5d539cbf4153a2dcbe7b28509fc6'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:06.996805 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-471e9dad-c979-43d7-87a5-217dd18e2b4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2fe5d539cbf4153a2dcbe7b28509fc6'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:06.997469 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-471e9dad-c979-43d7-87a5-217dd18e2b4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2fe5d539cbf4153a2dcbe7b28509fc6'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:06.998704 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-471e9dad-c979-43d7-87a5-217dd18e2b4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2fe5d539cbf4153a2dcbe7b28509fc6'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:07.010591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-471e9dad-c979-43d7-87a5-217dd18e2b4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2fe5d539cbf4153a2dcbe7b28509fc6'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:07.024869 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1144/2261] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:06 2022] DELETE /identity/v3/users/f2fe5d539cbf4153a2dcbe7b28509fc6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:07.029931 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eaf49d-4f83-4794-9998-20f2ce69d19d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:07.030244 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eaf49d-4f83-4794-9998-20f2ce69d19d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:07.030521 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eaf49d-4f83-4794-9998-20f2ce69d19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:07.050127 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-55eaf49d-4f83-4794-9998-20f2ce69d19d None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:07.051345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-55eaf49d-4f83-4794-9998-20f2ce69d19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:07.109136 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1118/2262] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:34:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:07.119113 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e9dad-c979-43d7-87a5-217dd18e2b4d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:07.120559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-471e9dad-c979-43d7-87a5-217dd18e2b4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:07.178000 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e9dad-c979-43d7-87a5-217dd18e2b4d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:07.181665 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f32b15-ae17-49f5-aa51-201b976f82a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:07.182299 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f32b15-ae17-49f5-aa51-201b976f82a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:07.182584 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f32b15-ae17-49f5-aa51-201b976f82a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:07.182858 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f32b15-ae17-49f5-aa51-201b976f82a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:07.191766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f32b15-ae17-49f5-aa51-201b976f82a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:07.193184 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f32b15-ae17-49f5-aa51-201b976f82a9 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:07.203034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1145/2263] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:07.469893 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eaf49d-4f83-4794-9998-20f2ce69d19d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:07.471423 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-55eaf49d-4f83-4794-9998-20f2ce69d19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:07.483574 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eaf49d-4f83-4794-9998-20f2ce69d19d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:07.488322 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc45081-9d22-44f9-837f-ed4ad041658c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:07.489084 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc45081-9d22-44f9-837f-ed4ad041658c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:07.489360 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc45081-9d22-44f9-837f-ed4ad041658c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:07.489637 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc45081-9d22-44f9-837f-ed4ad041658c admin admin] PATH_INFO: `/v3/projects/77395891bff3497e9a8e063a4e3be13d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:07.490930 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc45081-9d22-44f9-837f-ed4ad041658c admin admin] RBAC: Authorizing `identity:delete_project(project_id=77395891bff3497e9a8e063a4e3be13d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:07.493024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc45081-9d22-44f9-837f-ed4ad041658c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:07.540721 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4dc45081-9d22-44f9-837f-ed4ad041658c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77395891bff3497e9a8e063a4e3be13d', 'request_id': 'req-4dc45081-9d22-44f9-837f-ed4ad041658c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:07.546832 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4dc45081-9d22-44f9-837f-ed4ad041658c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77395891bff3497e9a8e063a4e3be13d', 'request_id': 'req-4dc45081-9d22-44f9-837f-ed4ad041658c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:07.554294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4dc45081-9d22-44f9-837f-ed4ad041658c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77395891bff3497e9a8e063a4e3be13d', 'request_id': 'req-4dc45081-9d22-44f9-837f-ed4ad041658c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:07.565714 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4dc45081-9d22-44f9-837f-ed4ad041658c admin admin] The token cache is being invalidate because project 77395891bff3497e9a8e063a4e3be13d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:07.566002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4dc45081-9d22-44f9-837f-ed4ad041658c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:07.566658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4dc45081-9d22-44f9-837f-ed4ad041658c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:07.568130 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1119/2264] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:07 2022] DELETE /identity/v3/projects/77395891bff3497e9a8e063a4e3be13d => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:17.198397 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:17.562527 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:40.095173 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20518ab2-a583-4f6e-b320-35243ec2b3c1 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:40.095698 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20518ab2-a583-4f6e-b320-35243ec2b3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:40.095783 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20518ab2-a583-4f6e-b320-35243ec2b3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:40.116226 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-20518ab2-a583-4f6e-b320-35243ec2b3c1 None None] MFA Rules not processed for user `024adeb8a8e14d678ae14174d9555152`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:40.117422 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-20518ab2-a583-4f6e-b320-35243ec2b3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:40.173973 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1146/2265] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:34:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:41.291255 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f98679-61d8-4c6f-b9d8-78899cb79d0a None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:41.291782 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f98679-61d8-4c6f-b9d8-78899cb79d0a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:41.291881 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f98679-61d8-4c6f-b9d8-78899cb79d0a None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:41.293154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1120/2266] 10.222.0.58 () {54 vars in 961 bytes} [Thu Sep 1 16:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:34:41.298842 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220f32f-7bb1-48b8-9c33-12586fa65ded None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:41.299313 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220f32f-7bb1-48b8-9c33-12586fa65ded None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:41.299540 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220f32f-7bb1-48b8-9c33-12586fa65ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:41.321175 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-8220f32f-7bb1-48b8-9c33-12586fa65ded None None] MFA Rules not processed for user `e12302cdb2854b0684179647a6dadf6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:41.322333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8220f32f-7bb1-48b8-9c33-12586fa65ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:41.388054 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1147/2267] 10.222.0.58 () {58 vars in 1058 bytes} [Thu Sep 1 16:34:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:41.395391 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f98679-61d8-4c6f-b9d8-78899cb79d0a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:41.409056 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f98679-61d8-4c6f-b9d8-78899cb79d0a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:41.414035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b672f22-c04f-4035-b225-e409cc9864fc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12302cdb2854b0684179647a6dadf6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:41.414672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b672f22-c04f-4035-b225-e409cc9864fc service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:41.414957 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b672f22-c04f-4035-b225-e409cc9864fc service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:41.415236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b672f22-c04f-4035-b225-e409cc9864fc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:41.416971 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7b672f22-c04f-4035-b225-e409cc9864fc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:41.429346 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b672f22-c04f-4035-b225-e409cc9864fc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:41.430972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b672f22-c04f-4035-b225-e409cc9864fc service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:41.441255 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1121/2268] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:51.379466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=54 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:51.437093 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:58.148785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ff557-77d8-4020-8585-6e90fac428a9 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:58.150016 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ff557-77d8-4020-8585-6e90fac428a9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:58.150861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ff557-77d8-4020-8585-6e90fac428a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:58.174021 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-102ff557-77d8-4020-8585-6e90fac428a9 None None] MFA Rules not processed for user `dd7b98536fa445cebd552fcbf54b757d`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:58.175254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-102ff557-77d8-4020-8585-6e90fac428a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:58.241458 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1148/2269] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Sep 1 16:34:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:34:58.285947 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b672f22-c04f-4035-b225-e409cc9864fc service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:58.287404 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7b672f22-c04f-4035-b225-e409cc9864fc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:58.360435 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b672f22-c04f-4035-b225-e409cc9864fc service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:58.365658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f845af6-3dff-4344-9d54-0a693d55ef5c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e789695517c1437fb224f5a6195655f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:58.366307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f845af6-3dff-4344-9d54-0a693d55ef5c service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:58.366593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f845af6-3dff-4344-9d54-0a693d55ef5c service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:58.366869 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f845af6-3dff-4344-9d54-0a693d55ef5c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:58.375464 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f845af6-3dff-4344-9d54-0a693d55ef5c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:58.377052 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f845af6-3dff-4344-9d54-0a693d55ef5c service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:58.385266 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1122/2270] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Sep 1 16:34:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:35:08.243873 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=25 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:08.382621 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=32 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:15.488701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de819944-78aa-4259-8f18-317ccabf144a None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:15.489279 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de819944-78aa-4259-8f18-317ccabf144a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:15.489379 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de819944-78aa-4259-8f18-317ccabf144a None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:15.490735 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1149/2271] 10.222.0.58 () {54 vars in 921 bytes} [Thu Sep 1 16:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:35:15.497336 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03577a2d-85fc-4644-8601-72893f86b17e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:15.497898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03577a2d-85fc-4644-8601-72893f86b17e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:15.497898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03577a2d-85fc-4644-8601-72893f86b17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:15.518655 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-03577a2d-85fc-4644-8601-72893f86b17e None None] MFA Rules not processed for user `1f0c99fdf8bd4331ad7e641ee38554c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:15.519927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-03577a2d-85fc-4644-8601-72893f86b17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:15.592058 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1123/2272] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Sep 1 16:35:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:15.603945 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de819944-78aa-4259-8f18-317ccabf144a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:15.605807 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-de819944-78aa-4259-8f18-317ccabf144a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:15.684449 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de819944-78aa-4259-8f18-317ccabf144a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:15.688505 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b83c70-1a51-4b79-82ae-9456fa3bceff service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2c872129414105a112661cc2b477e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:15.689117 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b83c70-1a51-4b79-82ae-9456fa3bceff service glance] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:15.689398 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b83c70-1a51-4b79-82ae-9456fa3bceff service glance] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:15.689674 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b83c70-1a51-4b79-82ae-9456fa3bceff service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:15.700391 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b83c70-1a51-4b79-82ae-9456fa3bceff service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:15.701821 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b83c70-1a51-4b79-82ae-9456fa3bceff service glance] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:15.712808 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1150/2273] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:15.785766 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e7f0c-e749-4bf8-aa94-26ded90eff32 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:15.786045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e7f0c-e749-4bf8-aa94-26ded90eff32 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:15.786338 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e7f0c-e749-4bf8-aa94-26ded90eff32 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:15.787684 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1124/2274] 10.222.0.58 () {54 vars in 921 bytes} [Thu Sep 1 16:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:35:15.792381 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a0520-c6ad-48e4-bc4d-d164d6b05e1c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:15.792671 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a0520-c6ad-48e4-bc4d-d164d6b05e1c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:15.793481 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a0520-c6ad-48e4-bc4d-d164d6b05e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:15.815325 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-523a0520-c6ad-48e4-bc4d-d164d6b05e1c None None] MFA Rules not processed for user `943fae29c5dc41f2911bbe051c74a361`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:15.816652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-523a0520-c6ad-48e4-bc4d-d164d6b05e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:15.872343 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1151/2275] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Sep 1 16:35:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:15.884112 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e7f0c-e749-4bf8-aa94-26ded90eff32 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:15.885870 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6a8e7f0c-e749-4bf8-aa94-26ded90eff32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:15.898906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e7f0c-e749-4bf8-aa94-26ded90eff32 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:15.902368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c6245c-7a7a-4d5a-966f-0a4133875841 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:15.902957 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6245c-7a7a-4d5a-966f-0a4133875841 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:15.903241 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6245c-7a7a-4d5a-966f-0a4133875841 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:15.903514 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6245c-7a7a-4d5a-966f-0a4133875841 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:15.910628 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c6245c-7a7a-4d5a-966f-0a4133875841 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:15.912041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c6245c-7a7a-4d5a-966f-0a4133875841 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:15.920304 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1125/2276] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:16.010772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc7331-a213-4d9b-8303-6c72e26975db None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:16.011065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc7331-a213-4d9b-8303-6c72e26975db None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:16.011341 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc7331-a213-4d9b-8303-6c72e26975db None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:16.013277 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1152/2277] 10.222.0.58 () {54 vars in 921 bytes} [Thu Sep 1 16:35:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:35:16.018784 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fff75d-f5e9-4f89-bc15-b729b8b79ba6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:16.019071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fff75d-f5e9-4f89-bc15-b729b8b79ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:16.019344 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fff75d-f5e9-4f89-bc15-b729b8b79ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:16.040034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-79fff75d-f5e9-4f89-bc15-b729b8b79ba6 None None] MFA Rules not processed for user `1f0c99fdf8bd4331ad7e641ee38554c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:16.041222 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-79fff75d-f5e9-4f89-bc15-b729b8b79ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:16.050953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1126/2278] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Sep 1 16:35:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:16.061754 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc7331-a213-4d9b-8303-6c72e26975db None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:16.073217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc7331-a213-4d9b-8303-6c72e26975db None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:16.077304 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88732d01-f730-40b9-8a41-bbc77b7f219a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:16.077905 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88732d01-f730-40b9-8a41-bbc77b7f219a service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:16.078273 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88732d01-f730-40b9-8a41-bbc77b7f219a service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:16.078551 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88732d01-f730-40b9-8a41-bbc77b7f219a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:16.088585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88732d01-f730-40b9-8a41-bbc77b7f219a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:16.090100 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88732d01-f730-40b9-8a41-bbc77b7f219a service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:16.103770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1153/2279] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:35:19.186390 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3c2f6-0cf8-40e0-b259-6593fa234ac0 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:19.187323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3c2f6-0cf8-40e0-b259-6593fa234ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:19.187809 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3c2f6-0cf8-40e0-b259-6593fa234ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:19.207717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-00a3c2f6-0cf8-40e0-b259-6593fa234ac0 None None] MFA Rules not processed for user `dd7b98536fa445cebd552fcbf54b757d`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:19.208933 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-00a3c2f6-0cf8-40e0-b259-6593fa234ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:19.216474 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1127/2280] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Sep 1 16:35:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:19.228746 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88732d01-f730-40b9-8a41-bbc77b7f219a service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:19.240700 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88732d01-f730-40b9-8a41-bbc77b7f219a service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:19.247714 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56418a2d-dd29-4f13-af2c-6ef899e689fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e789695517c1437fb224f5a6195655f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:19.248321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56418a2d-dd29-4f13-af2c-6ef899e689fe service swift] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:19.248601 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56418a2d-dd29-4f13-af2c-6ef899e689fe service swift] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:19.248879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56418a2d-dd29-4f13-af2c-6ef899e689fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:19.257398 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56418a2d-dd29-4f13-af2c-6ef899e689fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:19.259007 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56418a2d-dd29-4f13-af2c-6ef899e689fe service swift] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:19.268366 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1154/2281] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Sep 1 16:35:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:35:29.201999 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=43 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:29.264943 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=61 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:43.751107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3c2f6-0cf8-40e0-b259-6593fa234ac0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:43.760718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3c2f6-0cf8-40e0-b259-6593fa234ac0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:43.764318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c3e89f-34a8-4fa7-9936-d4e85459b252 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:43.764912 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3e89f-34a8-4fa7-9936-d4e85459b252 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:43.765191 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3e89f-34a8-4fa7-9936-d4e85459b252 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:43.765464 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3e89f-34a8-4fa7-9936-d4e85459b252 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:43.767196 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-71c3e89f-34a8-4fa7-9936-d4e85459b252 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:43.781024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c3e89f-34a8-4fa7-9936-d4e85459b252 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:43.782540 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c3e89f-34a8-4fa7-9936-d4e85459b252 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:43.793229 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1128/2282] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:44.978026 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56418a2d-dd29-4f13-af2c-6ef899e689fe service swift] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:44.995724 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56418a2d-dd29-4f13-af2c-6ef899e689fe service swift] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:44.999297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e1d8c-9fc6-4139-9469-d0162cc656c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:44.999887 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e1d8c-9fc6-4139-9469-d0162cc656c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:45.000161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e1d8c-9fc6-4139-9469-d0162cc656c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:45.000430 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e1d8c-9fc6-4139-9469-d0162cc656c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:45.002133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2d2e1d8c-9fc6-4139-9469-d0162cc656c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:45.015132 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e1d8c-9fc6-4139-9469-d0162cc656c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:45.016548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e1d8c-9fc6-4139-9469-d0162cc656c4 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:45.027985 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1155/2283] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:50.856294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abb993-b85c-481c-a382-a9e1e17c66b9 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:50.856764 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abb993-b85c-481c-a382-a9e1e17c66b9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:50.856880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abb993-b85c-481c-a382-a9e1e17c66b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:50.876307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-33abb993-b85c-481c-a382-a9e1e17c66b9 None None] MFA Rules not processed for user `dd7b98536fa445cebd552fcbf54b757d`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:50.877488 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-33abb993-b85c-481c-a382-a9e1e17c66b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:50.885776 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1129/2284] 10.222.0.58 () {58 vars in 967 bytes} [Thu Sep 1 16:35:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:50.898968 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e1d8c-9fc6-4139-9469-d0162cc656c4 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:50.910625 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e1d8c-9fc6-4139-9469-d0162cc656c4 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:50.915606 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9b547-15f5-48ff-8975-3427840b25ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e789695517c1437fb224f5a6195655f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:50.916222 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9b547-15f5-48ff-8975-3427840b25ee service swift] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:50.916508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9b547-15f5-48ff-8975-3427840b25ee service swift] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:50.916780 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9b547-15f5-48ff-8975-3427840b25ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:50.926123 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9b547-15f5-48ff-8975-3427840b25ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:50.927736 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9b547-15f5-48ff-8975-3427840b25ee service swift] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:50.937156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1156/2285] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Sep 1 16:35:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:35:59.295833 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33abb993-b85c-481c-a382-a9e1e17c66b9 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:59.297318 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-33abb993-b85c-481c-a382-a9e1e17c66b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:59.313895 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33abb993-b85c-481c-a382-a9e1e17c66b9 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:59.319292 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c6896-5b1b-4c22-81b1-a41aab5dc9f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:59.319885 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c6896-5b1b-4c22-81b1-a41aab5dc9f1 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:59.320168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c6896-5b1b-4c22-81b1-a41aab5dc9f1 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:59.320440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c6896-5b1b-4c22-81b1-a41aab5dc9f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:59.322108 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0c9c6896-5b1b-4c22-81b1-a41aab5dc9f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:59.381145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9c6896-5b1b-4c22-81b1-a41aab5dc9f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:59.382768 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9c6896-5b1b-4c22-81b1-a41aab5dc9f1 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:59.391639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1130/2286] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:59 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:00.933298 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=12 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:09.387630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=37 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:14.771404 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07037ea-6141-40cb-8d8e-f9373df4cde5 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:14.771939 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07037ea-6141-40cb-8d8e-f9373df4cde5 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:14.771939 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07037ea-6141-40cb-8d8e-f9373df4cde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:14.790693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-c07037ea-6141-40cb-8d8e-f9373df4cde5 None None] MFA Rules not processed for user `0d4f8f3a6d42456d8e5cfbe20e54687a`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:14.791895 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c07037ea-6141-40cb-8d8e-f9373df4cde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:14.798702 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1157/2287] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:36:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:15.148157 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c6896-5b1b-4c22-81b1-a41aab5dc9f1 service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:15.158323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c6896-5b1b-4c22-81b1-a41aab5dc9f1 service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:15.162167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74dbfa7-8eaa-447e-bb72-b9ffb219aa1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:15.162865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74dbfa7-8eaa-447e-bb72-b9ffb219aa1f service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:15.163152 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74dbfa7-8eaa-447e-bb72-b9ffb219aa1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:15.163430 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74dbfa7-8eaa-447e-bb72-b9ffb219aa1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:15.165308 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a74dbfa7-8eaa-447e-bb72-b9ffb219aa1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:15.175447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74dbfa7-8eaa-447e-bb72-b9ffb219aa1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:15.177177 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74dbfa7-8eaa-447e-bb72-b9ffb219aa1f service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:15.184939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1131/2288] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:24.300684 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07037ea-6141-40cb-8d8e-f9373df4cde5 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:24.314790 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07037ea-6141-40cb-8d8e-f9373df4cde5 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:24.320657 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f36826-b18e-4d14-83f2-526596bda8ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:24.321262 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f36826-b18e-4d14-83f2-526596bda8ad service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:24.321537 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f36826-b18e-4d14-83f2-526596bda8ad service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:24.321804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f36826-b18e-4d14-83f2-526596bda8ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:24.323721 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f4f36826-b18e-4d14-83f2-526596bda8ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:24.335867 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f36826-b18e-4d14-83f2-526596bda8ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:24.337953 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f36826-b18e-4d14-83f2-526596bda8ad service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:24.348280 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1158/2289] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:25.181535 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:34.343892 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=9 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:56.056899 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74dbfa7-8eaa-447e-bb72-b9ffb219aa1f service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:56.058317 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a74dbfa7-8eaa-447e-bb72-b9ffb219aa1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:56.074487 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74dbfa7-8eaa-447e-bb72-b9ffb219aa1f service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:56.079862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0e4b3-22a0-4e9b-bbf0-d81fb69e4a6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:56.080458 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0e4b3-22a0-4e9b-bbf0-d81fb69e4a6d service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:56.080739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0e4b3-22a0-4e9b-bbf0-d81fb69e4a6d service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:56.081014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0e4b3-22a0-4e9b-bbf0-d81fb69e4a6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:56.082722 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-39f0e4b3-22a0-4e9b-bbf0-d81fb69e4a6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:56.092677 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0e4b3-22a0-4e9b-bbf0-d81fb69e4a6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:56.094263 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0e4b3-22a0-4e9b-bbf0-d81fb69e4a6d service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:56.102998 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1132/2290] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:06.098987 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:10.867111 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f36826-b18e-4d14-83f2-526596bda8ad service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:10.878371 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f36826-b18e-4d14-83f2-526596bda8ad service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:10.881127 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00c9c1-d7eb-4b47-9931-d99f5aa18221 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:10.881632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00c9c1-d7eb-4b47-9931-d99f5aa18221 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:10.881759 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00c9c1-d7eb-4b47-9931-d99f5aa18221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:10.883951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f1aac9-5fa4-40ba-967d-b8e6658176f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:10.884536 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1aac9-5fa4-40ba-967d-b8e6658176f2 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:10.884815 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1aac9-5fa4-40ba-967d-b8e6658176f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:10.885183 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1aac9-5fa4-40ba-967d-b8e6658176f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:10.887064 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e7f1aac9-5fa4-40ba-967d-b8e6658176f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:10.902062 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-da00c9c1-d7eb-4b47-9931-d99f5aa18221 None None] MFA Rules not processed for user `024adeb8a8e14d678ae14174d9555152`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:10.903406 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-da00c9c1-d7eb-4b47-9931-d99f5aa18221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:10.910903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1133/2291] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:37:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:10.948326 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f1aac9-5fa4-40ba-967d-b8e6658176f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:10.949904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f1aac9-5fa4-40ba-967d-b8e6658176f2 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:10.959500 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1159/2292] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:11.254940 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00c9c1-d7eb-4b47-9931-d99f5aa18221 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:11.264696 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00c9c1-d7eb-4b47-9931-d99f5aa18221 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:11.268196 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9d306-7130-4fe6-9b79-8ebc86e4c352 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:11.268852 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9d306-7130-4fe6-9b79-8ebc86e4c352 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:11.269155 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9d306-7130-4fe6-9b79-8ebc86e4c352 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:11.269425 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9d306-7130-4fe6-9b79-8ebc86e4c352 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:11.271222 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5de9d306-7130-4fe6-9b79-8ebc86e4c352 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:11.282392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9d306-7130-4fe6-9b79-8ebc86e4c352 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:11.283899 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9d306-7130-4fe6-9b79-8ebc86e4c352 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:11.292540 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1134/2293] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:11.601526 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f1aac9-5fa4-40ba-967d-b8e6658176f2 service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:11.602989 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e7f1aac9-5fa4-40ba-967d-b8e6658176f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:11.618040 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f1aac9-5fa4-40ba-967d-b8e6658176f2 service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:11.623103 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90bfdc7-2305-409e-8b10-429cffe73b5f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12302cdb2854b0684179647a6dadf6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:11.623703 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90bfdc7-2305-409e-8b10-429cffe73b5f service placement] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:11.623985 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90bfdc7-2305-409e-8b10-429cffe73b5f service placement] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:11.624257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90bfdc7-2305-409e-8b10-429cffe73b5f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:11.625974 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e90bfdc7-2305-409e-8b10-429cffe73b5f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:11.637709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90bfdc7-2305-409e-8b10-429cffe73b5f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:11.639314 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90bfdc7-2305-409e-8b10-429cffe73b5f service placement] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:11.649239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1160/2294] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:21.288898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=9 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:21.645024 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=34 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:22.120536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9d306-7130-4fe6-9b79-8ebc86e4c352 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:22.134675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9d306-7130-4fe6-9b79-8ebc86e4c352 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:22.140076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69b844-2e15-4fa8-a5b1-e9e786f5eefe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:22.140675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69b844-2e15-4fa8-a5b1-e9e786f5eefe service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:22.140953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69b844-2e15-4fa8-a5b1-e9e786f5eefe service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:22.141226 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69b844-2e15-4fa8-a5b1-e9e786f5eefe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:22.143076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5f69b844-2e15-4fa8-a5b1-e9e786f5eefe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:22.156518 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69b844-2e15-4fa8-a5b1-e9e786f5eefe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:22.158123 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69b844-2e15-4fa8-a5b1-e9e786f5eefe service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:22.169498 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1135/2295] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:37:22.709232 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90bfdc7-2305-409e-8b10-429cffe73b5f service placement] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:22.721801 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90bfdc7-2305-409e-8b10-429cffe73b5f service placement] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:22.727400 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9c09e-0ed2-4d42-99e2-4bc5a0844582 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:22.728010 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9c09e-0ed2-4d42-99e2-4bc5a0844582 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:22.728289 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9c09e-0ed2-4d42-99e2-4bc5a0844582 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:22.728560 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9c09e-0ed2-4d42-99e2-4bc5a0844582 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:22.730314 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d1b9c09e-0ed2-4d42-99e2-4bc5a0844582 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:22.787426 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9c09e-0ed2-4d42-99e2-4bc5a0844582 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:22.788978 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9c09e-0ed2-4d42-99e2-4bc5a0844582 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:22.797720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1161/2296] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:22 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:32.165087 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:32.793669 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=28 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:33.451941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe5af4-35ae-4257-be2a-717edcbf6e86 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:33.452426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe5af4-35ae-4257-be2a-717edcbf6e86 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:33.452538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe5af4-35ae-4257-be2a-717edcbf6e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:33.472250 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-abfe5af4-35ae-4257-be2a-717edcbf6e86 None None] MFA Rules not processed for user `a328e946d6d4441ca7ecbb3a20d2a611`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:33.473443 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-abfe5af4-35ae-4257-be2a-717edcbf6e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:33.480423 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1136/2297] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:37:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:43.467033 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:00.693303 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9c09e-0ed2-4d42-99e2-4bc5a0844582 service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:00.706678 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9c09e-0ed2-4d42-99e2-4bc5a0844582 service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:00.712678 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bdb997-68c6-405b-bdcc-3dfc03c676e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:00.713608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bdb997-68c6-405b-bdcc-3dfc03c676e9 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:00.713888 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bdb997-68c6-405b-bdcc-3dfc03c676e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:00.714647 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bdb997-68c6-405b-bdcc-3dfc03c676e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:00.716312 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-53bdb997-68c6-405b-bdcc-3dfc03c676e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:00.730861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bdb997-68c6-405b-bdcc-3dfc03c676e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:00.732722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bdb997-68c6-405b-bdcc-3dfc03c676e9 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:00.745494 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1162/2298] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:10.740113 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=6 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:15.193804 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe5af4-35ae-4257-be2a-717edcbf6e86 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:15.195219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-abfe5af4-35ae-4257-be2a-717edcbf6e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:15.207780 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe5af4-35ae-4257-be2a-717edcbf6e86 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:15.211516 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ff904-f911-42e2-ba72-7122f2c6aeca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:15.212136 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ff904-f911-42e2-ba72-7122f2c6aeca service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:15.212473 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ff904-f911-42e2-ba72-7122f2c6aeca service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:15.213032 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ff904-f911-42e2-ba72-7122f2c6aeca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:15.214816 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9f1ff904-f911-42e2-ba72-7122f2c6aeca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:15.278692 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ff904-f911-42e2-ba72-7122f2c6aeca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:15.280230 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ff904-f911-42e2-ba72-7122f2c6aeca service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:15.290604 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1137/2299] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:18.510430 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d9f31-a2e1-4361-ab3d-69a48ae45e17 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:18.511416 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d9f31-a2e1-4361-ab3d-69a48ae45e17 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:18.511823 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d9f31-a2e1-4361-ab3d-69a48ae45e17 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:18.513675 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1163/2300] 10.222.0.58 () {54 vars in 921 bytes} [Thu Sep 1 16:38:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:38:18.522477 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b233d-2363-4237-a046-9bf8454923ef None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:18.522991 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b233d-2363-4237-a046-9bf8454923ef None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:18.523088 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b233d-2363-4237-a046-9bf8454923ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:18.545107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fb6b233d-2363-4237-a046-9bf8454923ef None None] MFA Rules not processed for user `e12302cdb2854b0684179647a6dadf6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:18.547614 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fb6b233d-2363-4237-a046-9bf8454923ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:18.560526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1138/2301] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Sep 1 16:38:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:18.575725 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d9f31-a2e1-4361-ab3d-69a48ae45e17 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:18.593733 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d9f31-a2e1-4361-ab3d-69a48ae45e17 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:18.595750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b233d-2363-4237-a046-9bf8454923ef None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:18.597603 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fb6b233d-2363-4237-a046-9bf8454923ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:18.601844 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3574d59-5877-4603-a3b7-8586225cad89 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12302cdb2854b0684179647a6dadf6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:18.602777 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3574d59-5877-4603-a3b7-8586225cad89 service placement] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:18.603187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3574d59-5877-4603-a3b7-8586225cad89 service placement] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:18.603543 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3574d59-5877-4603-a3b7-8586225cad89 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:18.613694 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3574d59-5877-4603-a3b7-8586225cad89 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:18.615310 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3574d59-5877-4603-a3b7-8586225cad89 service placement] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:18.619384 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b233d-2363-4237-a046-9bf8454923ef None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:18.624405 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968b623-b794-4884-8043-9fb4befb0dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:18.625111 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968b623-b794-4884-8043-9fb4befb0dd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:18.625369 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1164/2302] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:38:18.625483 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968b623-b794-4884-8043-9fb4befb0dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:18.625664 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968b623-b794-4884-8043-9fb4befb0dd3 admin admin] PATH_INFO: `/v3/users/0d4f8f3a6d42456d8e5cfbe20e54687a` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:18.627280 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968b623-b794-4884-8043-9fb4befb0dd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d4f8f3a6d42456d8e5cfbe20e54687a)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:18.629459 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968b623-b794-4884-8043-9fb4befb0dd3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:18.688483 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2968b623-b794-4884-8043-9fb4befb0dd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d4f8f3a6d42456d8e5cfbe20e54687a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:18.695575 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2968b623-b794-4884-8043-9fb4befb0dd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d4f8f3a6d42456d8e5cfbe20e54687a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:18.696313 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2968b623-b794-4884-8043-9fb4befb0dd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d4f8f3a6d42456d8e5cfbe20e54687a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:18.704915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2968b623-b794-4884-8043-9fb4befb0dd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d4f8f3a6d42456d8e5cfbe20e54687a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:18.716554 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2968b623-b794-4884-8043-9fb4befb0dd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d4f8f3a6d42456d8e5cfbe20e54687a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:18.722982 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1139/2303] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:38:18 2022] DELETE /identity/v3/users/0d4f8f3a6d42456d8e5cfbe20e54687a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:38:18.898547 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3574d59-5877-4603-a3b7-8586225cad89 service placement] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:18.916298 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3574d59-5877-4603-a3b7-8586225cad89 service placement] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:18.977550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4daac1-0e18-4c68-95fe-cc5380df1a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:18.978177 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4daac1-0e18-4c68-95fe-cc5380df1a2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:18.978487 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4daac1-0e18-4c68-95fe-cc5380df1a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:18.978760 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4daac1-0e18-4c68-95fe-cc5380df1a2d admin admin] PATH_INFO: `/v3/projects/27cd65c8d8044dddaf7a761697d216b4` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:18.979897 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4daac1-0e18-4c68-95fe-cc5380df1a2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=27cd65c8d8044dddaf7a761697d216b4)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:18.981791 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4daac1-0e18-4c68-95fe-cc5380df1a2d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:19.027679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3b4daac1-0e18-4c68-95fe-cc5380df1a2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27cd65c8d8044dddaf7a761697d216b4', 'request_id': 'req-3b4daac1-0e18-4c68-95fe-cc5380df1a2d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:19.034270 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3b4daac1-0e18-4c68-95fe-cc5380df1a2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27cd65c8d8044dddaf7a761697d216b4', 'request_id': 'req-3b4daac1-0e18-4c68-95fe-cc5380df1a2d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:19.039746 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3b4daac1-0e18-4c68-95fe-cc5380df1a2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27cd65c8d8044dddaf7a761697d216b4', 'request_id': 'req-3b4daac1-0e18-4c68-95fe-cc5380df1a2d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:19.052022 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3b4daac1-0e18-4c68-95fe-cc5380df1a2d admin admin] The token cache is being invalidate because project 27cd65c8d8044dddaf7a761697d216b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:38:19.052314 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3b4daac1-0e18-4c68-95fe-cc5380df1a2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:19.052932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3b4daac1-0e18-4c68-95fe-cc5380df1a2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:19.054476 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1165/2304] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:38:18 2022] DELETE /identity/v3/projects/27cd65c8d8044dddaf7a761697d216b4 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:19.059295 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa3ec9-ef1d-444c-85a5-a5ffa1681991 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:19.059584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa3ec9-ef1d-444c-85a5-a5ffa1681991 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:19.059863 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa3ec9-ef1d-444c-85a5-a5ffa1681991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:19.079757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b6aa3ec9-ef1d-444c-85a5-a5ffa1681991 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:19.080956 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b6aa3ec9-ef1d-444c-85a5-a5ffa1681991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:19.131861 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:19.142480 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1140/2305] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:38:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:19.150431 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1cac6-9864-4a6e-9b6f-3ff51fee21a1 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:19.150718 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1cac6-9864-4a6e-9b6f-3ff51fee21a1 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:19.151017 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1cac6-9864-4a6e-9b6f-3ff51fee21a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:19.171477 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-3bb1cac6-9864-4a6e-9b6f-3ff51fee21a1 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:19.172761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3bb1cac6-9864-4a6e-9b6f-3ff51fee21a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:19.180873 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1166/2306] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:38:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:38:19.184979 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa3ec9-ef1d-444c-85a5-a5ffa1681991 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:19.195352 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa3ec9-ef1d-444c-85a5-a5ffa1681991 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:19.199983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda7ead2-a700-411c-b06d-54587ff1fb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:19.200601 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda7ead2-a700-411c-b06d-54587ff1fb6a admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:19.200887 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda7ead2-a700-411c-b06d-54587ff1fb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:19.201164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda7ead2-a700-411c-b06d-54587ff1fb6a admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:19.202479 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda7ead2-a700-411c-b06d-54587ff1fb6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:38:19.202789 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda7ead2-a700-411c-b06d-54587ff1fb6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:19.204512 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda7ead2-a700-411c-b06d-54587ff1fb6a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:19.216395 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1141/2307] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:38:19.220593 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb1cac6-9864-4a6e-9b6f-3ff51fee21a1 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:19.229716 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb1cac6-9864-4a6e-9b6f-3ff51fee21a1 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:19.234462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd4059-62ee-4794-951f-b430bc43e71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:19.235044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd4059-62ee-4794-951f-b430bc43e71a admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:19.235323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd4059-62ee-4794-951f-b430bc43e71a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:19.235593 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd4059-62ee-4794-951f-b430bc43e71a admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:19.236249 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd4059-62ee-4794-951f-b430bc43e71a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:19.238131 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd4059-62ee-4794-951f-b430bc43e71a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:19.265316 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1167/2308] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:38:19 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:38:19.269410 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda7ead2-a700-411c-b06d-54587ff1fb6a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:19.280096 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda7ead2-a700-411c-b06d-54587ff1fb6a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:19.338179 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8e747-9a1e-4c6d-962f-9b016cd0fce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:19.338816 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8e747-9a1e-4c6d-962f-9b016cd0fce3 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:19.339097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8e747-9a1e-4c6d-962f-9b016cd0fce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:19.339370 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8e747-9a1e-4c6d-962f-9b016cd0fce3 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:19.340005 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8e747-9a1e-4c6d-962f-9b016cd0fce3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:19.341931 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8e747-9a1e-4c6d-962f-9b016cd0fce3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:19.362401 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1142/2309] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:38:19 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:38:19.366541 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd4059-62ee-4794-951f-b430bc43e71a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:19.377276 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd4059-62ee-4794-951f-b430bc43e71a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:19.381841 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64a70f-12c5-416c-9f41-8e7ebe4b33de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:19.382453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64a70f-12c5-416c-9f41-8e7ebe4b33de admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:19.382732 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64a70f-12c5-416c-9f41-8e7ebe4b33de admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:19.383000 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64a70f-12c5-416c-9f41-8e7ebe4b33de admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:19.383672 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d64a70f-12c5-416c-9f41-8e7ebe4b33de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:19.385256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d64a70f-12c5-416c-9f41-8e7ebe4b33de admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:19.395678 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1168/2310] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:38:19.399773 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8e747-9a1e-4c6d-962f-9b016cd0fce3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:19.409617 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8e747-9a1e-4c6d-962f-9b016cd0fce3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:19.414135 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e75d9-2152-4a6e-b087-afea40736294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:19.414750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e75d9-2152-4a6e-b087-afea40736294 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:19.415029 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e75d9-2152-4a6e-b087-afea40736294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:19.415301 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e75d9-2152-4a6e-b087-afea40736294 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:19.415940 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e75d9-2152-4a6e-b087-afea40736294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:19.417522 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e75d9-2152-4a6e-b087-afea40736294 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:19.427891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1143/2311] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:38:19.431897 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64a70f-12c5-416c-9f41-8e7ebe4b33de admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:19.442172 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64a70f-12c5-416c-9f41-8e7ebe4b33de admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:19.446737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ba8d8-6a6f-4205-a3ac-6949e21d8591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:19.447370 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ba8d8-6a6f-4205-a3ac-6949e21d8591 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:19.447650 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ba8d8-6a6f-4205-a3ac-6949e21d8591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:19.447923 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ba8d8-6a6f-4205-a3ac-6949e21d8591 admin admin] PATH_INFO: `/v3/projects/de60d2ff06fc4b82935894af92698309/users/660eed406c814d1b9d9fae82631104b3/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:19.462679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ba8d8-6a6f-4205-a3ac-6949e21d8591 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de60d2ff06fc4b82935894af92698309, user_id=660eed406c814d1b9d9fae82631104b3, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:19.465309 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ba8d8-6a6f-4205-a3ac-6949e21d8591 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:19.482159 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1169/2312] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:38:19 2022] PUT /identity/v3/projects/de60d2ff06fc4b82935894af92698309/users/660eed406c814d1b9d9fae82631104b3/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:19.494324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71aa67-e241-42c7-be94-e67f894303f5 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:19.494614 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71aa67-e241-42c7-be94-e67f894303f5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:19.494890 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71aa67-e241-42c7-be94-e67f894303f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:19.523329 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8e71aa67-e241-42c7-be94-e67f894303f5 None None] MFA Rules not processed for user `660eed406c814d1b9d9fae82631104b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:19.524790 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8e71aa67-e241-42c7-be94-e67f894303f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:19.591749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1144/2313] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:38:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:19.596551 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ba8d8-6a6f-4205-a3ac-6949e21d8591 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:19.610592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:19.660397 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ba8d8-6a6f-4205-a3ac-6949e21d8591 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:19.665026 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5214443f-5fd9-400d-84c0-03fca5fdb0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:19.665620 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214443f-5fd9-400d-84c0-03fca5fdb0ae admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:19.665901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214443f-5fd9-400d-84c0-03fca5fdb0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:19.666177 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214443f-5fd9-400d-84c0-03fca5fdb0ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:19.666831 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5214443f-5fd9-400d-84c0-03fca5fdb0ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:19.668721 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5214443f-5fd9-400d-84c0-03fca5fdb0ae admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:19.694297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1170/2314] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:38:19 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:38:19.698298 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71aa67-e241-42c7-be94-e67f894303f5 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:19.709310 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71aa67-e241-42c7-be94-e67f894303f5 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:19.758301 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:19.769910 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add41972-e3f9-4fd6-a810-53969f5c71ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:19.771324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add41972-e3f9-4fd6-a810-53969f5c71ae admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:19.771705 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add41972-e3f9-4fd6-a810-53969f5c71ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:19.772166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add41972-e3f9-4fd6-a810-53969f5c71ae admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:19.773133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add41972-e3f9-4fd6-a810-53969f5c71ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:19.775748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add41972-e3f9-4fd6-a810-53969f5c71ae admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:19.797752 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1145/2315] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:38:19 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:38:19.801766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5214443f-5fd9-400d-84c0-03fca5fdb0ae admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:19.812243 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5214443f-5fd9-400d-84c0-03fca5fdb0ae admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:19.816956 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2255c56-c0ad-417b-9837-b712124b8303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:19.817579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2255c56-c0ad-417b-9837-b712124b8303 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:19.817908 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2255c56-c0ad-417b-9837-b712124b8303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:19.818132 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2255c56-c0ad-417b-9837-b712124b8303 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:19.818816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2255c56-c0ad-417b-9837-b712124b8303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:19.820398 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2255c56-c0ad-417b-9837-b712124b8303 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:19.832028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1171/2316] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:38:19.836370 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add41972-e3f9-4fd6-a810-53969f5c71ae admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:19.846948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add41972-e3f9-4fd6-a810-53969f5c71ae admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:19.851493 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add93a4-a269-42ed-ab5f-4ea767f2bfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:19.852134 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add93a4-a269-42ed-ab5f-4ea767f2bfe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:19.852412 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add93a4-a269-42ed-ab5f-4ea767f2bfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:19.852684 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add93a4-a269-42ed-ab5f-4ea767f2bfe5 admin admin] PATH_INFO: `/v3/projects/53d24a7834e24d899e3f38832200386b/users/9d5f24d2f90f4882be4a19047d724f9d/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:19.866928 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5add93a4-a269-42ed-ab5f-4ea767f2bfe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53d24a7834e24d899e3f38832200386b, user_id=9d5f24d2f90f4882be4a19047d724f9d, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:19.869573 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5add93a4-a269-42ed-ab5f-4ea767f2bfe5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:19.881852 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1146/2317] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:38:19 2022] PUT /identity/v3/projects/53d24a7834e24d899e3f38832200386b/users/9d5f24d2f90f4882be4a19047d724f9d/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:19.885940 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2255c56-c0ad-417b-9837-b712124b8303 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:19.896840 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2255c56-c0ad-417b-9837-b712124b8303 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:19.954492 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3f0e6-bc38-4917-b4d1-8e9b11c1dd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:19.955099 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3f0e6-bc38-4917-b4d1-8e9b11c1dd39 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:19.955392 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3f0e6-bc38-4917-b4d1-8e9b11c1dd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:19.955677 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3f0e6-bc38-4917-b4d1-8e9b11c1dd39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:19.956347 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c3f0e6-bc38-4917-b4d1-8e9b11c1dd39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:19.958027 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c3f0e6-bc38-4917-b4d1-8e9b11c1dd39 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:19.968339 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1172/2318] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:38:19.972243 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add93a4-a269-42ed-ab5f-4ea767f2bfe5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:19.982365 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add93a4-a269-42ed-ab5f-4ea767f2bfe5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:19.986913 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365ad2a-f638-402a-b308-cfd97c84bc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:19.987553 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365ad2a-f638-402a-b308-cfd97c84bc16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:19.987837 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365ad2a-f638-402a-b308-cfd97c84bc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:19.988113 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365ad2a-f638-402a-b308-cfd97c84bc16 admin admin] PATH_INFO: `/v3/projects/53d24a7834e24d899e3f38832200386b/users/9d5f24d2f90f4882be4a19047d724f9d/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:19.990007 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365ad2a-f638-402a-b308-cfd97c84bc16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53d24a7834e24d899e3f38832200386b, user_id=9d5f24d2f90f4882be4a19047d724f9d, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:19.992651 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365ad2a-f638-402a-b308-cfd97c84bc16 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:20.003478 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1147/2319] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:38:19 2022] PUT /identity/v3/projects/53d24a7834e24d899e3f38832200386b/users/9d5f24d2f90f4882be4a19047d724f9d/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:20.013441 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc77edc-55e4-490a-98d0-88a2daefd3a3 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:20.013757 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc77edc-55e4-490a-98d0-88a2daefd3a3 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:20.014673 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc77edc-55e4-490a-98d0-88a2daefd3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:20.043115 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5dc77edc-55e4-490a-98d0-88a2daefd3a3 None None] MFA Rules not processed for user `9d5f24d2f90f4882be4a19047d724f9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:20.044325 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5dc77edc-55e4-490a-98d0-88a2daefd3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:20.112502 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1173/2320] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:38:20 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:20.124146 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365ad2a-f638-402a-b308-cfd97c84bc16 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:20.125850 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f365ad2a-f638-402a-b308-cfd97c84bc16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:20.182745 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365ad2a-f638-402a-b308-cfd97c84bc16 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:20.186858 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ad968-344e-4b94-88d9-83f9f27346d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:20.187446 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ad968-344e-4b94-88d9-83f9f27346d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:20.187724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ad968-344e-4b94-88d9-83f9f27346d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:20.187995 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ad968-344e-4b94-88d9-83f9f27346d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:20.196751 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ad968-344e-4b94-88d9-83f9f27346d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:20.198167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ad968-344e-4b94-88d9-83f9f27346d4 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:20.208010 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1148/2321] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:20 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:20.284949 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc77edc-55e4-490a-98d0-88a2daefd3a3 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:20.294427 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc77edc-55e4-490a-98d0-88a2daefd3a3 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:20.297998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac11ad-2334-48ad-bc10-d99116b3cf89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:20.298634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac11ad-2334-48ad-bc10-d99116b3cf89 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:20.298912 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac11ad-2334-48ad-bc10-d99116b3cf89 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:20.299184 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac11ad-2334-48ad-bc10-d99116b3cf89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:20.304196 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:20.358860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac11ad-2334-48ad-bc10-d99116b3cf89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:20.360281 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac11ad-2334-48ad-bc10-d99116b3cf89 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:20.369102 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1174/2322] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:30.203983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=45,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:30.364997 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=24 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:33.326182 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ad968-344e-4b94-88d9-83f9f27346d4 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:33.389760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ad968-344e-4b94-88d9-83f9f27346d4 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:33.394367 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d2bb4-4c29-4ad4-ad82-436c23415367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:33.394985 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d2bb4-4c29-4ad4-ad82-436c23415367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:33.395266 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d2bb4-4c29-4ad4-ad82-436c23415367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:33.395547 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d2bb4-4c29-4ad4-ad82-436c23415367 admin admin] PATH_INFO: `/v3/users/660eed406c814d1b9d9fae82631104b3` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:33.396656 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d2bb4-4c29-4ad4-ad82-436c23415367 admin admin] RBAC: Authorizing `identity:delete_user(user_id=660eed406c814d1b9d9fae82631104b3)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:33.398838 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d2bb4-4c29-4ad4-ad82-436c23415367 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:33.448387 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-690d2bb4-4c29-4ad4-ad82-436c23415367 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '660eed406c814d1b9d9fae82631104b3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:33.455972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-690d2bb4-4c29-4ad4-ad82-436c23415367 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '660eed406c814d1b9d9fae82631104b3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:33.456646 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-690d2bb4-4c29-4ad4-ad82-436c23415367 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '660eed406c814d1b9d9fae82631104b3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:33.463830 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-690d2bb4-4c29-4ad4-ad82-436c23415367 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '660eed406c814d1b9d9fae82631104b3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:33.476413 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-690d2bb4-4c29-4ad4-ad82-436c23415367 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '660eed406c814d1b9d9fae82631104b3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:33.482852 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1149/2323] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:38:33 2022] DELETE /identity/v3/users/660eed406c814d1b9d9fae82631104b3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:33.487425 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706ff0f-2ab3-4a94-802b-2713265863bd None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:33.487934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706ff0f-2ab3-4a94-802b-2713265863bd None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:33.488037 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706ff0f-2ab3-4a94-802b-2713265863bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:33.506861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a706ff0f-2ab3-4a94-802b-2713265863bd None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:33.508039 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a706ff0f-2ab3-4a94-802b-2713265863bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:33.565956 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1175/2324] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:38:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:33.575526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d2bb4-4c29-4ad4-ad82-436c23415367 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:33.576919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-690d2bb4-4c29-4ad4-ad82-436c23415367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:33.634980 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d2bb4-4c29-4ad4-ad82-436c23415367 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:33.639571 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b31b2-b18d-4f5d-ba75-78bd02fc3cd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:33.640175 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b31b2-b18d-4f5d-ba75-78bd02fc3cd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:33.640458 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b31b2-b18d-4f5d-ba75-78bd02fc3cd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:33.640730 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b31b2-b18d-4f5d-ba75-78bd02fc3cd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:33.650957 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72b31b2-b18d-4f5d-ba75-78bd02fc3cd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:33.652386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72b31b2-b18d-4f5d-ba75-78bd02fc3cd3 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:33.662183 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1150/2325] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:34.144826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706ff0f-2ab3-4a94-802b-2713265863bd None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:34.146496 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a706ff0f-2ab3-4a94-802b-2713265863bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:34.158700 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706ff0f-2ab3-4a94-802b-2713265863bd None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:34.163348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105e6f7-43d5-4b8c-a799-6d8e22021370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:34.163945 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105e6f7-43d5-4b8c-a799-6d8e22021370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:34.164224 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105e6f7-43d5-4b8c-a799-6d8e22021370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:34.164504 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105e6f7-43d5-4b8c-a799-6d8e22021370 admin admin] PATH_INFO: `/v3/projects/de60d2ff06fc4b82935894af92698309` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:34.165621 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4105e6f7-43d5-4b8c-a799-6d8e22021370 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de60d2ff06fc4b82935894af92698309)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:34.167583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4105e6f7-43d5-4b8c-a799-6d8e22021370 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:34.214312 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4105e6f7-43d5-4b8c-a799-6d8e22021370 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de60d2ff06fc4b82935894af92698309', 'request_id': 'req-4105e6f7-43d5-4b8c-a799-6d8e22021370'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:34.220842 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4105e6f7-43d5-4b8c-a799-6d8e22021370 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de60d2ff06fc4b82935894af92698309', 'request_id': 'req-4105e6f7-43d5-4b8c-a799-6d8e22021370'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:34.226920 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4105e6f7-43d5-4b8c-a799-6d8e22021370 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de60d2ff06fc4b82935894af92698309', 'request_id': 'req-4105e6f7-43d5-4b8c-a799-6d8e22021370'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:34.239137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4105e6f7-43d5-4b8c-a799-6d8e22021370 admin admin] The token cache is being invalidate because project de60d2ff06fc4b82935894af92698309 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:38:34.239450 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4105e6f7-43d5-4b8c-a799-6d8e22021370 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:34.240067 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4105e6f7-43d5-4b8c-a799-6d8e22021370 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:34.241790 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1176/2326] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:38:34 2022] DELETE /identity/v3/projects/de60d2ff06fc4b82935894af92698309 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:34.245778 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b31b2-b18d-4f5d-ba75-78bd02fc3cd3 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:34.255283 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b31b2-b18d-4f5d-ba75-78bd02fc3cd3 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:34.272801 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:34.313060 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8fe53-eabf-41b5-b991-24f13c2eba01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:34.313612 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8fe53-eabf-41b5-b991-24f13c2eba01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:34.313895 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8fe53-eabf-41b5-b991-24f13c2eba01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:34.314170 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8fe53-eabf-41b5-b991-24f13c2eba01 admin admin] PATH_INFO: `/v3/users/9d5f24d2f90f4882be4a19047d724f9d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:34.315313 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8fe53-eabf-41b5-b991-24f13c2eba01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d5f24d2f90f4882be4a19047d724f9d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:34.317241 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8fe53-eabf-41b5-b991-24f13c2eba01 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:34.362566 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f2b8fe53-eabf-41b5-b991-24f13c2eba01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d5f24d2f90f4882be4a19047d724f9d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:34.369255 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f2b8fe53-eabf-41b5-b991-24f13c2eba01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d5f24d2f90f4882be4a19047d724f9d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:34.369900 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f2b8fe53-eabf-41b5-b991-24f13c2eba01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d5f24d2f90f4882be4a19047d724f9d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:34.377658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f2b8fe53-eabf-41b5-b991-24f13c2eba01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d5f24d2f90f4882be4a19047d724f9d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:34.389833 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f2b8fe53-eabf-41b5-b991-24f13c2eba01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d5f24d2f90f4882be4a19047d724f9d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:34.395550 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1151/2327] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:38:34 2022] DELETE /identity/v3/users/9d5f24d2f90f4882be4a19047d724f9d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:38:34.663826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105e6f7-43d5-4b8c-a799-6d8e22021370 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:34.665273 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4105e6f7-43d5-4b8c-a799-6d8e22021370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:34.678250 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105e6f7-43d5-4b8c-a799-6d8e22021370 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:34.736706 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2c624-0069-453d-b6d7-09768c8b0fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:34.737340 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2c624-0069-453d-b6d7-09768c8b0fee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:34.737621 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2c624-0069-453d-b6d7-09768c8b0fee admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:34.737892 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2c624-0069-453d-b6d7-09768c8b0fee admin admin] PATH_INFO: `/v3/projects/53d24a7834e24d899e3f38832200386b` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:34.739057 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2c624-0069-453d-b6d7-09768c8b0fee admin admin] RBAC: Authorizing `identity:delete_project(project_id=53d24a7834e24d899e3f38832200386b)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:34.740992 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2c624-0069-453d-b6d7-09768c8b0fee admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:34.771575 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:34.789521 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ade2c624-0069-453d-b6d7-09768c8b0fee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53d24a7834e24d899e3f38832200386b', 'request_id': 'req-ade2c624-0069-453d-b6d7-09768c8b0fee'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:34.796209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ade2c624-0069-453d-b6d7-09768c8b0fee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53d24a7834e24d899e3f38832200386b', 'request_id': 'req-ade2c624-0069-453d-b6d7-09768c8b0fee'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:34.802562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ade2c624-0069-453d-b6d7-09768c8b0fee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53d24a7834e24d899e3f38832200386b', 'request_id': 'req-ade2c624-0069-453d-b6d7-09768c8b0fee'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:34.817208 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ade2c624-0069-453d-b6d7-09768c8b0fee admin admin] The token cache is being invalidate because project 53d24a7834e24d899e3f38832200386b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:38:34.817507 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ade2c624-0069-453d-b6d7-09768c8b0fee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:34.818130 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ade2c624-0069-453d-b6d7-09768c8b0fee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:34.819811 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1177/2328] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:38:34 2022] DELETE /identity/v3/projects/53d24a7834e24d899e3f38832200386b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:34.824749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263255f-13f4-44b7-ae01-7dd73a283337 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:34.825034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263255f-13f4-44b7-ae01-7dd73a283337 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:34.825310 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263255f-13f4-44b7-ae01-7dd73a283337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:34.845862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7263255f-13f4-44b7-ae01-7dd73a283337 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:34.847013 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7263255f-13f4-44b7-ae01-7dd73a283337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:34.905824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1152/2329] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:38:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:34.915054 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447a954-3ba0-4755-adfa-69abb5b77ebf None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:34.915340 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447a954-3ba0-4755-adfa-69abb5b77ebf None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:34.915615 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447a954-3ba0-4755-adfa-69abb5b77ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:34.935423 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e447a954-3ba0-4755-adfa-69abb5b77ebf None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:34.936605 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e447a954-3ba0-4755-adfa-69abb5b77ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:34.944361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1178/2330] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:38:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:34.948623 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263255f-13f4-44b7-ae01-7dd73a283337 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:34.958903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263255f-13f4-44b7-ae01-7dd73a283337 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:34.963413 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d30f1-2fb6-4c25-87e7-0fc5e5caa38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:34.964002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d30f1-2fb6-4c25-87e7-0fc5e5caa38b admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:34.964282 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d30f1-2fb6-4c25-87e7-0fc5e5caa38b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:34.964555 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d30f1-2fb6-4c25-87e7-0fc5e5caa38b admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:34.965202 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d30f1-2fb6-4c25-87e7-0fc5e5caa38b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:38:34.965506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d30f1-2fb6-4c25-87e7-0fc5e5caa38b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:34.967179 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d30f1-2fb6-4c25-87e7-0fc5e5caa38b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:34.978464 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1153/2331] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:38:34.982702 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447a954-3ba0-4755-adfa-69abb5b77ebf None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:34.991703 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447a954-3ba0-4755-adfa-69abb5b77ebf None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:34.996287 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d30a3a-5675-4a7d-b4fe-1bec1dc8c989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:34.996870 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d30a3a-5675-4a7d-b4fe-1bec1dc8c989 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:34.997147 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d30a3a-5675-4a7d-b4fe-1bec1dc8c989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:34.997922 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d30a3a-5675-4a7d-b4fe-1bec1dc8c989 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:34.998575 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d30a3a-5675-4a7d-b4fe-1bec1dc8c989 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:35.000439 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d30a3a-5675-4a7d-b4fe-1bec1dc8c989 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:35.028399 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1179/2332] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:38:34 2022] POST /identity/v3/projects => generated 357 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:38:35.033233 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d30f1-2fb6-4c25-87e7-0fc5e5caa38b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:35.043194 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d30f1-2fb6-4c25-87e7-0fc5e5caa38b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:35.069976 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:35.102007 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bfcafc-d676-456e-9b9a-6843d1d909ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:35.102574 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfcafc-d676-456e-9b9a-6843d1d909ca admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:35.102857 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfcafc-d676-456e-9b9a-6843d1d909ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:35.103132 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfcafc-d676-456e-9b9a-6843d1d909ca admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:35.103769 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bfcafc-d676-456e-9b9a-6843d1d909ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:35.105729 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bfcafc-d676-456e-9b9a-6843d1d909ca admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:35.126563 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1154/2333] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:38:35 2022] POST /identity/v3/users => generated 279 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:38:35.130867 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d30a3a-5675-4a7d-b4fe-1bec1dc8c989 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:35.144339 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d30a3a-5675-4a7d-b4fe-1bec1dc8c989 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:35.149231 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51240f-bf05-4571-bd71-55bf622235c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:35.149808 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51240f-bf05-4571-bd71-55bf622235c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:35.150091 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51240f-bf05-4571-bd71-55bf622235c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:35.150400 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51240f-bf05-4571-bd71-55bf622235c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:35.151048 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51240f-bf05-4571-bd71-55bf622235c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:35.152668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51240f-bf05-4571-bd71-55bf622235c3 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:35.163341 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1180/2334] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:38:35.167202 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bfcafc-d676-456e-9b9a-6843d1d909ca admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:35.177644 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bfcafc-d676-456e-9b9a-6843d1d909ca admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:35.182228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa4ef5-c9e0-4807-bc4a-8cb08d22efe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:35.182818 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa4ef5-c9e0-4807-bc4a-8cb08d22efe5 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:35.183100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa4ef5-c9e0-4807-bc4a-8cb08d22efe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:35.183373 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa4ef5-c9e0-4807-bc4a-8cb08d22efe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:35.184013 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa4ef5-c9e0-4807-bc4a-8cb08d22efe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:35.185638 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa4ef5-c9e0-4807-bc4a-8cb08d22efe5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:35.196022 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1155/2335] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:38:35.199863 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51240f-bf05-4571-bd71-55bf622235c3 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:35.209723 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51240f-bf05-4571-bd71-55bf622235c3 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:35.214286 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c3ffb-b57b-4b4d-b596-d0a912097dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:35.214921 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c3ffb-b57b-4b4d-b596-d0a912097dcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:35.215206 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c3ffb-b57b-4b4d-b596-d0a912097dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:35.215479 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c3ffb-b57b-4b4d-b596-d0a912097dcd admin admin] PATH_INFO: `/v3/projects/4a8c2c369bc643fa9c2a7c7d43fa213c/users/84dfc99e5769438b989527fb2a3e72fc/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:35.230135 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c3ffb-b57b-4b4d-b596-d0a912097dcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a8c2c369bc643fa9c2a7c7d43fa213c, user_id=84dfc99e5769438b989527fb2a3e72fc, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:35.233443 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c3ffb-b57b-4b4d-b596-d0a912097dcd admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:35.246027 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1181/2336] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:38:35 2022] PUT /identity/v3/projects/4a8c2c369bc643fa9c2a7c7d43fa213c/users/84dfc99e5769438b989527fb2a3e72fc/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:38:35.256442 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef96f1-5537-42bf-bbf4-74abce8b91dd None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:35.256729 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef96f1-5537-42bf-bbf4-74abce8b91dd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:35.257005 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef96f1-5537-42bf-bbf4-74abce8b91dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:35.284753 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-53ef96f1-5537-42bf-bbf4-74abce8b91dd None None] MFA Rules not processed for user `84dfc99e5769438b989527fb2a3e72fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:35.286023 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-53ef96f1-5537-42bf-bbf4-74abce8b91dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:35.350920 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1156/2337] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:38:35 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:35.355488 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c3ffb-b57b-4b4d-b596-d0a912097dcd admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:35.416656 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c3ffb-b57b-4b4d-b596-d0a912097dcd admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:35.421211 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d18a48-9054-46d3-99f7-d93cc35100a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:35.421804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d18a48-9054-46d3-99f7-d93cc35100a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:35.422086 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d18a48-9054-46d3-99f7-d93cc35100a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:35.422395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d18a48-9054-46d3-99f7-d93cc35100a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:35.423030 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d18a48-9054-46d3-99f7-d93cc35100a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:35.424972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d18a48-9054-46d3-99f7-d93cc35100a9 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:35.450984 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1182/2338] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:38:35 2022] POST /identity/v3/projects => generated 357 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:38:35.454860 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef96f1-5537-42bf-bbf4-74abce8b91dd None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:35.465607 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef96f1-5537-42bf-bbf4-74abce8b91dd None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:35.520159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0e84e-a700-45c8-af80-03565707ad51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:35.520751 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0e84e-a700-45c8-af80-03565707ad51 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:35.521029 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0e84e-a700-45c8-af80-03565707ad51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:35.521300 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0e84e-a700-45c8-af80-03565707ad51 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:35.521941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c0e84e-a700-45c8-af80-03565707ad51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:35.524134 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c0e84e-a700-45c8-af80-03565707ad51 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:35.544672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1157/2339] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:38:35 2022] POST /identity/v3/users => generated 285 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:38:35.548705 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d18a48-9054-46d3-99f7-d93cc35100a9 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:35.558681 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d18a48-9054-46d3-99f7-d93cc35100a9 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:35.563912 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a11c7-d4c3-478f-a48e-f0b7d67ddc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:35.564495 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a11c7-d4c3-478f-a48e-f0b7d67ddc96 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:35.564772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a11c7-d4c3-478f-a48e-f0b7d67ddc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:35.565041 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a11c7-d4c3-478f-a48e-f0b7d67ddc96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:35.565674 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10a11c7-d4c3-478f-a48e-f0b7d67ddc96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:35.567296 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10a11c7-d4c3-478f-a48e-f0b7d67ddc96 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:35.578582 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1183/2340] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:38:35.582614 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0e84e-a700-45c8-af80-03565707ad51 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:35.592531 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0e84e-a700-45c8-af80-03565707ad51 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:35.597105 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a25f0c-8976-43b2-8967-24af5b3b06a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:35.597741 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a25f0c-8976-43b2-8967-24af5b3b06a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:35.598024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a25f0c-8976-43b2-8967-24af5b3b06a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:35.598333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a25f0c-8976-43b2-8967-24af5b3b06a0 admin admin] PATH_INFO: `/v3/projects/2065a036bc0944f583806e558c678fd9/users/9964ea3073df4b04839715af59311972/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:35.613941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a25f0c-8976-43b2-8967-24af5b3b06a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2065a036bc0944f583806e558c678fd9, user_id=9964ea3073df4b04839715af59311972, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:35.616574 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a25f0c-8976-43b2-8967-24af5b3b06a0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:35.629518 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1158/2341] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:38:35 2022] PUT /identity/v3/projects/2065a036bc0944f583806e558c678fd9/users/9964ea3073df4b04839715af59311972/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:35.633381 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a11c7-d4c3-478f-a48e-f0b7d67ddc96 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:35.643217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a11c7-d4c3-478f-a48e-f0b7d67ddc96 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:35.662713 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:35.707703 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf7967-569a-4733-9580-c3f6c7b1ccd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:35.708357 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf7967-569a-4733-9580-c3f6c7b1ccd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:35.708587 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf7967-569a-4733-9580-c3f6c7b1ccd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:35.708858 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf7967-569a-4733-9580-c3f6c7b1ccd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:35.709551 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdf7967-569a-4733-9580-c3f6c7b1ccd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:35.711164 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdf7967-569a-4733-9580-c3f6c7b1ccd4 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:35.722078 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1184/2342] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:38:35.726993 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a25f0c-8976-43b2-8967-24af5b3b06a0 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:35.733290 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:35.739319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a25f0c-8976-43b2-8967-24af5b3b06a0 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:35.744536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1849e-3f4d-4410-9366-e4d4c19a8a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:35.745192 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1849e-3f4d-4410-9366-e4d4c19a8a29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:35.745479 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1849e-3f4d-4410-9366-e4d4c19a8a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:35.745757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1849e-3f4d-4410-9366-e4d4c19a8a29 admin admin] PATH_INFO: `/v3/projects/2065a036bc0944f583806e558c678fd9/users/9964ea3073df4b04839715af59311972/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:35.747698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe1849e-3f4d-4410-9366-e4d4c19a8a29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2065a036bc0944f583806e558c678fd9, user_id=9964ea3073df4b04839715af59311972, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:35.750489 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe1849e-3f4d-4410-9366-e4d4c19a8a29 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:35.760496 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1159/2343] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:38:35 2022] PUT /identity/v3/projects/2065a036bc0944f583806e558c678fd9/users/9964ea3073df4b04839715af59311972/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:35.770745 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f73b03-d8a9-4109-9d8d-ac3e3535922c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:35.771097 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f73b03-d8a9-4109-9d8d-ac3e3535922c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:35.771467 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f73b03-d8a9-4109-9d8d-ac3e3535922c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:35.800692 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f4f73b03-d8a9-4109-9d8d-ac3e3535922c None None] MFA Rules not processed for user `9964ea3073df4b04839715af59311972`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:35.801904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f4f73b03-d8a9-4109-9d8d-ac3e3535922c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:35.873557 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1185/2344] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:38:35 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:35.885230 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1849e-3f4d-4410-9366-e4d4c19a8a29 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:35.886969 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bbe1849e-3f4d-4410-9366-e4d4c19a8a29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:35.943574 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1849e-3f4d-4410-9366-e4d4c19a8a29 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:35.947236 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d48fdf-c65e-42da-aad3-43827e972c2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:35.947833 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d48fdf-c65e-42da-aad3-43827e972c2d service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:35.948113 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d48fdf-c65e-42da-aad3-43827e972c2d service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:35.948385 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d48fdf-c65e-42da-aad3-43827e972c2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:35.957109 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d48fdf-c65e-42da-aad3-43827e972c2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:35.958569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d48fdf-c65e-42da-aad3-43827e972c2d service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:35.968199 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1160/2345] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:35 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:36.043860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f73b03-d8a9-4109-9d8d-ac3e3535922c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:36.054564 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f73b03-d8a9-4109-9d8d-ac3e3535922c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:36.058202 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dcf60e-f876-47df-ad6f-526499822d88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:36.058811 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcf60e-f876-47df-ad6f-526499822d88 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:36.059087 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcf60e-f876-47df-ad6f-526499822d88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:36.059357 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcf60e-f876-47df-ad6f-526499822d88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:36.120059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dcf60e-f876-47df-ad6f-526499822d88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:36.121489 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dcf60e-f876-47df-ad6f-526499822d88 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:36.130956 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1186/2346] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:36 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:45.963593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:46.126634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=80 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:52.123912 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d48fdf-c65e-42da-aad3-43827e972c2d service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:52.245517 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d48fdf-c65e-42da-aad3-43827e972c2d service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:52.250640 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd7b96-8769-416b-a3e7-c099ccbecd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:52.251325 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd7b96-8769-416b-a3e7-c099ccbecd26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:52.251611 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd7b96-8769-416b-a3e7-c099ccbecd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:52.251886 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd7b96-8769-416b-a3e7-c099ccbecd26 admin admin] PATH_INFO: `/v3/users/84dfc99e5769438b989527fb2a3e72fc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:52.253035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd7b96-8769-416b-a3e7-c099ccbecd26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84dfc99e5769438b989527fb2a3e72fc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:52.255022 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd7b96-8769-416b-a3e7-c099ccbecd26 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:52.300697 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1fdd7b96-8769-416b-a3e7-c099ccbecd26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84dfc99e5769438b989527fb2a3e72fc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:52.307377 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1fdd7b96-8769-416b-a3e7-c099ccbecd26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84dfc99e5769438b989527fb2a3e72fc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:52.308026 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1fdd7b96-8769-416b-a3e7-c099ccbecd26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84dfc99e5769438b989527fb2a3e72fc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:52.315334 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1fdd7b96-8769-416b-a3e7-c099ccbecd26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84dfc99e5769438b989527fb2a3e72fc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:52.328259 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1fdd7b96-8769-416b-a3e7-c099ccbecd26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84dfc99e5769438b989527fb2a3e72fc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:52.334131 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1161/2347] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:38:52 2022] DELETE /identity/v3/users/84dfc99e5769438b989527fb2a3e72fc => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:52.338588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8397eb-56ed-48cd-9e4d-9027b0b3d440 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:52.339057 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8397eb-56ed-48cd-9e4d-9027b0b3d440 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:52.339169 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8397eb-56ed-48cd-9e4d-9027b0b3d440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:52.387700 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-4c8397eb-56ed-48cd-9e4d-9027b0b3d440 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:52.388820 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4c8397eb-56ed-48cd-9e4d-9027b0b3d440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:52.446014 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1187/2348] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:52.455749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd7b96-8769-416b-a3e7-c099ccbecd26 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:52.457127 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1fdd7b96-8769-416b-a3e7-c099ccbecd26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:52.503403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:52.545506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd7b96-8769-416b-a3e7-c099ccbecd26 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:52.549178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcc2f9a-6191-44dd-a53c-6a676bbcef8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:52.549907 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc2f9a-6191-44dd-a53c-6a676bbcef8d service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:52.550210 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc2f9a-6191-44dd-a53c-6a676bbcef8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:52.550499 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc2f9a-6191-44dd-a53c-6a676bbcef8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:52.560969 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcc2f9a-6191-44dd-a53c-6a676bbcef8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:52.562455 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcc2f9a-6191-44dd-a53c-6a676bbcef8d service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:52.571766 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1162/2349] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:52.847687 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8397eb-56ed-48cd-9e4d-9027b0b3d440 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:52.849179 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4c8397eb-56ed-48cd-9e4d-9027b0b3d440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:52.862088 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8397eb-56ed-48cd-9e4d-9027b0b3d440 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:52.866900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39592e8b-72a6-4dcc-8d53-0d93d4f2aa88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:52.867629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39592e8b-72a6-4dcc-8d53-0d93d4f2aa88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:52.868015 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39592e8b-72a6-4dcc-8d53-0d93d4f2aa88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:52.868355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39592e8b-72a6-4dcc-8d53-0d93d4f2aa88 admin admin] PATH_INFO: `/v3/projects/4a8c2c369bc643fa9c2a7c7d43fa213c` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:52.869620 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39592e8b-72a6-4dcc-8d53-0d93d4f2aa88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a8c2c369bc643fa9c2a7c7d43fa213c)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:52.871627 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39592e8b-72a6-4dcc-8d53-0d93d4f2aa88 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:52.918866 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-39592e8b-72a6-4dcc-8d53-0d93d4f2aa88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a8c2c369bc643fa9c2a7c7d43fa213c', 'request_id': 'req-39592e8b-72a6-4dcc-8d53-0d93d4f2aa88'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:52.924873 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-39592e8b-72a6-4dcc-8d53-0d93d4f2aa88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a8c2c369bc643fa9c2a7c7d43fa213c', 'request_id': 'req-39592e8b-72a6-4dcc-8d53-0d93d4f2aa88'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:52.930373 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-39592e8b-72a6-4dcc-8d53-0d93d4f2aa88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a8c2c369bc643fa9c2a7c7d43fa213c', 'request_id': 'req-39592e8b-72a6-4dcc-8d53-0d93d4f2aa88'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:52.942473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-39592e8b-72a6-4dcc-8d53-0d93d4f2aa88 admin admin] The token cache is being invalidate because project 4a8c2c369bc643fa9c2a7c7d43fa213c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:38:52.942759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-39592e8b-72a6-4dcc-8d53-0d93d4f2aa88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:52.943442 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-39592e8b-72a6-4dcc-8d53-0d93d4f2aa88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:52.944985 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1188/2350] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:38:52 2022] DELETE /identity/v3/projects/4a8c2c369bc643fa9c2a7c7d43fa213c => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:52.948945 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcc2f9a-6191-44dd-a53c-6a676bbcef8d service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:52.958622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcc2f9a-6191-44dd-a53c-6a676bbcef8d service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:53.014680 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd09eb-b1d7-449d-93ce-c101e866f08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:53.015303 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd09eb-b1d7-449d-93ce-c101e866f08b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:53.015581 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd09eb-b1d7-449d-93ce-c101e866f08b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:53.015852 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd09eb-b1d7-449d-93ce-c101e866f08b admin admin] PATH_INFO: `/v3/users/9964ea3073df4b04839715af59311972` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:53.016953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd09eb-b1d7-449d-93ce-c101e866f08b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9964ea3073df4b04839715af59311972)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:53.018897 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd09eb-b1d7-449d-93ce-c101e866f08b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:53.067240 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-48dd09eb-b1d7-449d-93ce-c101e866f08b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9964ea3073df4b04839715af59311972'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:53.074632 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-48dd09eb-b1d7-449d-93ce-c101e866f08b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9964ea3073df4b04839715af59311972'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:53.075298 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-48dd09eb-b1d7-449d-93ce-c101e866f08b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9964ea3073df4b04839715af59311972'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:53.082629 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-48dd09eb-b1d7-449d-93ce-c101e866f08b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9964ea3073df4b04839715af59311972'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:53.095181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-48dd09eb-b1d7-449d-93ce-c101e866f08b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9964ea3073df4b04839715af59311972'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:53.100939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1163/2351] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:38:52 2022] DELETE /identity/v3/users/9964ea3073df4b04839715af59311972 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:53.363700 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39592e8b-72a6-4dcc-8d53-0d93d4f2aa88 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:53.365222 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-39592e8b-72a6-4dcc-8d53-0d93d4f2aa88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:53.378855 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39592e8b-72a6-4dcc-8d53-0d93d4f2aa88 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:53.430237 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:53.437664 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0ad10-d87c-4dfd-a285-98af0abb8022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:53.438538 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0ad10-d87c-4dfd-a285-98af0abb8022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:53.439027 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0ad10-d87c-4dfd-a285-98af0abb8022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:53.439476 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0ad10-d87c-4dfd-a285-98af0abb8022 admin admin] PATH_INFO: `/v3/projects/2065a036bc0944f583806e558c678fd9` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:53.440832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e0ad10-d87c-4dfd-a285-98af0abb8022 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2065a036bc0944f583806e558c678fd9)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:53.442978 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e0ad10-d87c-4dfd-a285-98af0abb8022 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:53.488663 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-54e0ad10-d87c-4dfd-a285-98af0abb8022 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2065a036bc0944f583806e558c678fd9', 'request_id': 'req-54e0ad10-d87c-4dfd-a285-98af0abb8022'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:53.494638 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-54e0ad10-d87c-4dfd-a285-98af0abb8022 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2065a036bc0944f583806e558c678fd9', 'request_id': 'req-54e0ad10-d87c-4dfd-a285-98af0abb8022'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:53.500807 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-54e0ad10-d87c-4dfd-a285-98af0abb8022 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2065a036bc0944f583806e558c678fd9', 'request_id': 'req-54e0ad10-d87c-4dfd-a285-98af0abb8022'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:53.513496 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-54e0ad10-d87c-4dfd-a285-98af0abb8022 admin admin] The token cache is being invalidate because project 2065a036bc0944f583806e558c678fd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:38:53.513788 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-54e0ad10-d87c-4dfd-a285-98af0abb8022 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:53.514475 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-54e0ad10-d87c-4dfd-a285-98af0abb8022 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:53.515947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1189/2352] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:38:53 2022] DELETE /identity/v3/projects/2065a036bc0944f583806e558c678fd9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:53.520823 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93afe5-1123-41f7-a966-d95a0a1cfb6d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:53.521306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93afe5-1123-41f7-a966-d95a0a1cfb6d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:53.521414 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93afe5-1123-41f7-a966-d95a0a1cfb6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:53.570989 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7d93afe5-1123-41f7-a966-d95a0a1cfb6d None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:53.572177 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7d93afe5-1123-41f7-a966-d95a0a1cfb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:53.585122 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:53.638297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1164/2353] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:53.646651 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b2d05-c413-453d-862e-91cb8abde879 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:53.647032 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b2d05-c413-453d-862e-91cb8abde879 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:53.647631 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b2d05-c413-453d-862e-91cb8abde879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:53.667137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-fe2b2d05-c413-453d-862e-91cb8abde879 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:53.668340 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-fe2b2d05-c413-453d-862e-91cb8abde879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:53.676254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1190/2354] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:53.680459 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93afe5-1123-41f7-a966-d95a0a1cfb6d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:53.690691 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93afe5-1123-41f7-a966-d95a0a1cfb6d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:53.695230 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d91853-51cc-4583-af87-fdd3186d8274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:53.695820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d91853-51cc-4583-af87-fdd3186d8274 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:53.696099 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d91853-51cc-4583-af87-fdd3186d8274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:53.696371 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d91853-51cc-4583-af87-fdd3186d8274 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:53.697010 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d91853-51cc-4583-af87-fdd3186d8274 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:38:53.697312 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d91853-51cc-4583-af87-fdd3186d8274 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:53.699569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d91853-51cc-4583-af87-fdd3186d8274 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:53.711067 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1165/2355] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:38:53.715347 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2b2d05-c413-453d-862e-91cb8abde879 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:53.724491 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2b2d05-c413-453d-862e-91cb8abde879 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:53.729047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248e758-e17c-4f07-9eb0-dd7cd7b35b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:53.729631 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248e758-e17c-4f07-9eb0-dd7cd7b35b22 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:53.729912 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248e758-e17c-4f07-9eb0-dd7cd7b35b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:53.730183 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248e758-e17c-4f07-9eb0-dd7cd7b35b22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:53.730837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248e758-e17c-4f07-9eb0-dd7cd7b35b22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:53.732706 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248e758-e17c-4f07-9eb0-dd7cd7b35b22 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:53.759133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1191/2356] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:38:53 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:38:53.763982 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d91853-51cc-4583-af87-fdd3186d8274 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:53.774183 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d91853-51cc-4583-af87-fdd3186d8274 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:53.830015 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e3f58-9e84-4f26-8e25-5be19e4db331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:53.830649 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e3f58-9e84-4f26-8e25-5be19e4db331 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:53.830932 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e3f58-9e84-4f26-8e25-5be19e4db331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:53.831208 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e3f58-9e84-4f26-8e25-5be19e4db331 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:53.831866 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e3f58-9e84-4f26-8e25-5be19e4db331 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:53.833765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e3f58-9e84-4f26-8e25-5be19e4db331 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:53.854561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1166/2357] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:38:53 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:38:53.859166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248e758-e17c-4f07-9eb0-dd7cd7b35b22 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:53.870757 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248e758-e17c-4f07-9eb0-dd7cd7b35b22 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:53.878456 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbc007-9b37-4a53-b1ea-605f3d4e335a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:53.879069 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbc007-9b37-4a53-b1ea-605f3d4e335a admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:53.879351 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbc007-9b37-4a53-b1ea-605f3d4e335a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:53.879807 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbc007-9b37-4a53-b1ea-605f3d4e335a admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:53.880471 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efbc007-9b37-4a53-b1ea-605f3d4e335a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:53.882071 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efbc007-9b37-4a53-b1ea-605f3d4e335a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:53.892578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1192/2358] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:38:53.896630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e3f58-9e84-4f26-8e25-5be19e4db331 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:53.906910 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e3f58-9e84-4f26-8e25-5be19e4db331 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:53.911428 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968aa8fc-f209-47ff-8abe-f66255416573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:53.912014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968aa8fc-f209-47ff-8abe-f66255416573 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:53.912295 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968aa8fc-f209-47ff-8abe-f66255416573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:53.912567 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968aa8fc-f209-47ff-8abe-f66255416573 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:53.913208 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968aa8fc-f209-47ff-8abe-f66255416573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:53.914836 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968aa8fc-f209-47ff-8abe-f66255416573 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:53.925629 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1167/2359] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:38:53.929553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbc007-9b37-4a53-b1ea-605f3d4e335a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:53.940008 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbc007-9b37-4a53-b1ea-605f3d4e335a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:53.944583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67001d4e-e0b4-4bcd-b7f7-6f07778076a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:53.945219 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67001d4e-e0b4-4bcd-b7f7-6f07778076a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:53.945497 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67001d4e-e0b4-4bcd-b7f7-6f07778076a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:53.945769 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67001d4e-e0b4-4bcd-b7f7-6f07778076a8 admin admin] PATH_INFO: `/v3/projects/054c2935d14045639844e73344ea71d9/users/c836cf3ddc3946baab137c15ded9cbe8/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:53.961333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67001d4e-e0b4-4bcd-b7f7-6f07778076a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=054c2935d14045639844e73344ea71d9, user_id=c836cf3ddc3946baab137c15ded9cbe8, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:53.964001 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67001d4e-e0b4-4bcd-b7f7-6f07778076a8 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:53.976499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1193/2360] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:38:53 2022] PUT /identity/v3/projects/054c2935d14045639844e73344ea71d9/users/c836cf3ddc3946baab137c15ded9cbe8/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:53.987603 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ea23f-9e72-4d0f-b677-e4cad7c3b912 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:53.987889 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ea23f-9e72-4d0f-b677-e4cad7c3b912 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:53.988162 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ea23f-9e72-4d0f-b677-e4cad7c3b912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:54.015850 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2c9ea23f-9e72-4d0f-b677-e4cad7c3b912 None None] MFA Rules not processed for user `c836cf3ddc3946baab137c15ded9cbe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:54.016975 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2c9ea23f-9e72-4d0f-b677-e4cad7c3b912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:54.082110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1168/2361] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:38:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:54.086552 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67001d4e-e0b4-4bcd-b7f7-6f07778076a8 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:54.147097 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67001d4e-e0b4-4bcd-b7f7-6f07778076a8 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:54.152018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4549583e-124d-453e-afb3-20a053f7d28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:54.152607 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549583e-124d-453e-afb3-20a053f7d28e admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:54.152894 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549583e-124d-453e-afb3-20a053f7d28e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:54.153168 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549583e-124d-453e-afb3-20a053f7d28e admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:54.153801 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4549583e-124d-453e-afb3-20a053f7d28e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:54.155752 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4549583e-124d-453e-afb3-20a053f7d28e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:54.181631 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1194/2362] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:38:54 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:38:54.186664 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ea23f-9e72-4d0f-b677-e4cad7c3b912 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:54.197528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ea23f-9e72-4d0f-b677-e4cad7c3b912 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:54.209358 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:54.255642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea9dff-ce1b-4704-9917-db4fe9fbfdd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:54.256253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea9dff-ce1b-4704-9917-db4fe9fbfdd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:54.256537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea9dff-ce1b-4704-9917-db4fe9fbfdd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:54.256813 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea9dff-ce1b-4704-9917-db4fe9fbfdd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:54.257438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea9dff-ce1b-4704-9917-db4fe9fbfdd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:54.259386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea9dff-ce1b-4704-9917-db4fe9fbfdd0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:54.280933 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1169/2363] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:38:54 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:38:54.284885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4549583e-124d-453e-afb3-20a053f7d28e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:54.295923 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4549583e-124d-453e-afb3-20a053f7d28e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:54.300403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63badaa4-02ba-409f-a68a-f38113891c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:54.300991 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63badaa4-02ba-409f-a68a-f38113891c3b admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:54.301274 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63badaa4-02ba-409f-a68a-f38113891c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:54.301562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63badaa4-02ba-409f-a68a-f38113891c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:54.302242 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63badaa4-02ba-409f-a68a-f38113891c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:54.303817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63badaa4-02ba-409f-a68a-f38113891c3b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:54.305814 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:54.316011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1195/2364] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:38:54.319760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea9dff-ce1b-4704-9917-db4fe9fbfdd0 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:54.330640 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea9dff-ce1b-4704-9917-db4fe9fbfdd0 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:54.335483 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46afffc-0c05-4266-a714-9f0185694e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:54.336173 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46afffc-0c05-4266-a714-9f0185694e1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:54.336459 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46afffc-0c05-4266-a714-9f0185694e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:54.336734 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46afffc-0c05-4266-a714-9f0185694e1a admin admin] PATH_INFO: `/v3/projects/cb8039e1d29a491eb68f8df1e81c51f8/users/388beff30a9f4a3997fddc518ddd910c/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:54.351969 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46afffc-0c05-4266-a714-9f0185694e1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb8039e1d29a491eb68f8df1e81c51f8, user_id=388beff30a9f4a3997fddc518ddd910c, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:54.354684 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46afffc-0c05-4266-a714-9f0185694e1a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:54.367644 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1170/2365] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:38:54 2022] PUT /identity/v3/projects/cb8039e1d29a491eb68f8df1e81c51f8/users/388beff30a9f4a3997fddc518ddd910c/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:54.371906 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63badaa4-02ba-409f-a68a-f38113891c3b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:54.382018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63badaa4-02ba-409f-a68a-f38113891c3b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:54.437979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8f057-9a49-43a8-b4ef-b4c099a5e0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:54.438615 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8f057-9a49-43a8-b4ef-b4c099a5e0b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:54.438891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8f057-9a49-43a8-b4ef-b4c099a5e0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:54.439162 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8f057-9a49-43a8-b4ef-b4c099a5e0b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:54.439802 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8f057-9a49-43a8-b4ef-b4c099a5e0b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:54.441384 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8f057-9a49-43a8-b4ef-b4c099a5e0b1 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:54.450787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1196/2366] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:38:54.455232 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46afffc-0c05-4266-a714-9f0185694e1a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:54.465847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46afffc-0c05-4266-a714-9f0185694e1a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:54.470479 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a5203-e295-419d-96ef-961ebc0828fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:54.471125 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a5203-e295-419d-96ef-961ebc0828fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:54.471408 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a5203-e295-419d-96ef-961ebc0828fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:54.471683 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a5203-e295-419d-96ef-961ebc0828fd admin admin] PATH_INFO: `/v3/projects/cb8039e1d29a491eb68f8df1e81c51f8/users/388beff30a9f4a3997fddc518ddd910c/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:54.473672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a5203-e295-419d-96ef-961ebc0828fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb8039e1d29a491eb68f8df1e81c51f8, user_id=388beff30a9f4a3997fddc518ddd910c, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:54.476334 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a5203-e295-419d-96ef-961ebc0828fd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:54.486986 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1171/2367] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:38:54 2022] PUT /identity/v3/projects/cb8039e1d29a491eb68f8df1e81c51f8/users/388beff30a9f4a3997fddc518ddd910c/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:54.498223 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ffbad-310f-4c07-b3ce-154e5fc80f16 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:54.498568 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ffbad-310f-4c07-b3ce-154e5fc80f16 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:54.498848 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ffbad-310f-4c07-b3ce-154e5fc80f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:54.526304 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e64ffbad-310f-4c07-b3ce-154e5fc80f16 None None] MFA Rules not processed for user `388beff30a9f4a3997fddc518ddd910c`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:54.527999 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e64ffbad-310f-4c07-b3ce-154e5fc80f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:54.595630 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1197/2368] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:38:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:54.607188 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a5203-e295-419d-96ef-961ebc0828fd admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:54.608871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e85a5203-e295-419d-96ef-961ebc0828fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:54.665419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a5203-e295-419d-96ef-961ebc0828fd admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:54.669190 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd5961-805d-451c-8a8d-011bd287c47f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:54.669794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd5961-805d-451c-8a8d-011bd287c47f service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:54.670161 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd5961-805d-451c-8a8d-011bd287c47f service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:54.670469 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd5961-805d-451c-8a8d-011bd287c47f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:54.726005 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd5961-805d-451c-8a8d-011bd287c47f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:54.727520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd5961-805d-451c-8a8d-011bd287c47f service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:54.736280 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1172/2369] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:39:02.462734 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ffbad-310f-4c07-b3ce-154e5fc80f16 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:02.464772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e64ffbad-310f-4c07-b3ce-154e5fc80f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:02.476496 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ffbad-310f-4c07-b3ce-154e5fc80f16 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:02.480051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3edf6a-d545-4cdc-88d7-0cc48767e275 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:02.480685 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3edf6a-d545-4cdc-88d7-0cc48767e275 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:02.480965 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3edf6a-d545-4cdc-88d7-0cc48767e275 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:02.481239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3edf6a-d545-4cdc-88d7-0cc48767e275 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:02.489973 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3edf6a-d545-4cdc-88d7-0cc48767e275 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:02.491439 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3edf6a-d545-4cdc-88d7-0cc48767e275 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:02.501693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1198/2370] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:02 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:39:04.732341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=85,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:12.497440 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=70 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:33.961323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddfa49-e7bc-4fe3-8b09-33567b1d2a25 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:33.962439 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddfa49-e7bc-4fe3-8b09-33567b1d2a25 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:33.962577 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddfa49-e7bc-4fe3-8b09-33567b1d2a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:33.981405 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b5ddfa49-e7bc-4fe3-8b09-33567b1d2a25 None None] MFA Rules not processed for user `4ecfc51400414558b8389a9c35b2fa1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:33.982644 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b5ddfa49-e7bc-4fe3-8b09-33567b1d2a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:34.038539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1173/2371] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:39:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:34.359838 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3edf6a-d545-4cdc-88d7-0cc48767e275 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:34.361328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0a3edf6a-d545-4cdc-88d7-0cc48767e275 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:34.443454 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3edf6a-d545-4cdc-88d7-0cc48767e275 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:34.447522 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2fa987-10e0-46fd-8957-228ba301b59a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:34.448132 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fa987-10e0-46fd-8957-228ba301b59a service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:34.448417 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fa987-10e0-46fd-8957-228ba301b59a service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:34.448691 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fa987-10e0-46fd-8957-228ba301b59a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:34.450457 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7a2fa987-10e0-46fd-8957-228ba301b59a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:34.462569 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2fa987-10e0-46fd-8957-228ba301b59a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:34.465198 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2fa987-10e0-46fd-8957-228ba301b59a service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:34.474524 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1199/2372] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:39:35.626133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ddfa49-e7bc-4fe3-8b09-33567b1d2a25 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:35.687228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ddfa49-e7bc-4fe3-8b09-33567b1d2a25 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:35.691629 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625a641-6301-4898-9d41-648e7af4d4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:35.692266 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625a641-6301-4898-9d41-648e7af4d4ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:35.692553 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625a641-6301-4898-9d41-648e7af4d4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:35.692827 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625a641-6301-4898-9d41-648e7af4d4ca admin admin] PATH_INFO: `/v3/users/c836cf3ddc3946baab137c15ded9cbe8` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:35.693926 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625a641-6301-4898-9d41-648e7af4d4ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=c836cf3ddc3946baab137c15ded9cbe8)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:35.695964 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625a641-6301-4898-9d41-648e7af4d4ca admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:35.746500 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9625a641-6301-4898-9d41-648e7af4d4ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c836cf3ddc3946baab137c15ded9cbe8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:35.753213 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9625a641-6301-4898-9d41-648e7af4d4ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c836cf3ddc3946baab137c15ded9cbe8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:35.754034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9625a641-6301-4898-9d41-648e7af4d4ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c836cf3ddc3946baab137c15ded9cbe8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:35.761762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9625a641-6301-4898-9d41-648e7af4d4ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c836cf3ddc3946baab137c15ded9cbe8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:35.773935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9625a641-6301-4898-9d41-648e7af4d4ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c836cf3ddc3946baab137c15ded9cbe8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:35.779737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1174/2373] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:39:35 2022] DELETE /identity/v3/users/c836cf3ddc3946baab137c15ded9cbe8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:39:35.784213 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac37945-5b3e-463c-b98a-e2be629c80fc None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:35.784703 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac37945-5b3e-463c-b98a-e2be629c80fc None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:35.784811 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac37945-5b3e-463c-b98a-e2be629c80fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:35.809849 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-fac37945-5b3e-463c-b98a-e2be629c80fc None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:35.811540 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-fac37945-5b3e-463c-b98a-e2be629c80fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:35.885119 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1200/2374] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:39:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:35.895259 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625a641-6301-4898-9d41-648e7af4d4ca admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:35.896939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9625a641-6301-4898-9d41-648e7af4d4ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:35.925431 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:35.962458 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625a641-6301-4898-9d41-648e7af4d4ca admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:35.966139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45fb17-60d7-4d40-9a07-a47608943df8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:35.966775 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45fb17-60d7-4d40-9a07-a47608943df8 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:35.967060 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45fb17-60d7-4d40-9a07-a47608943df8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:35.967338 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45fb17-60d7-4d40-9a07-a47608943df8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:35.976296 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba45fb17-60d7-4d40-9a07-a47608943df8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:35.978261 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba45fb17-60d7-4d40-9a07-a47608943df8 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:35.987571 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1175/2375] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:39:36.300121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac37945-5b3e-463c-b98a-e2be629c80fc None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:36.301381 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-fac37945-5b3e-463c-b98a-e2be629c80fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:36.321937 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac37945-5b3e-463c-b98a-e2be629c80fc None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:36.328742 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f15a7-35d5-4ee6-aed2-20966e810c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:36.329134 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f15a7-35d5-4ee6-aed2-20966e810c61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:36.329420 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f15a7-35d5-4ee6-aed2-20966e810c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:36.329694 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f15a7-35d5-4ee6-aed2-20966e810c61 admin admin] PATH_INFO: `/v3/projects/054c2935d14045639844e73344ea71d9` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:36.330877 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f15a7-35d5-4ee6-aed2-20966e810c61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=054c2935d14045639844e73344ea71d9)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:36.332803 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f15a7-35d5-4ee6-aed2-20966e810c61 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:36.393156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d96f15a7-35d5-4ee6-aed2-20966e810c61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '054c2935d14045639844e73344ea71d9', 'request_id': 'req-d96f15a7-35d5-4ee6-aed2-20966e810c61'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:36.399502 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d96f15a7-35d5-4ee6-aed2-20966e810c61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '054c2935d14045639844e73344ea71d9', 'request_id': 'req-d96f15a7-35d5-4ee6-aed2-20966e810c61'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:36.408141 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d96f15a7-35d5-4ee6-aed2-20966e810c61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '054c2935d14045639844e73344ea71d9', 'request_id': 'req-d96f15a7-35d5-4ee6-aed2-20966e810c61'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:36.423077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d96f15a7-35d5-4ee6-aed2-20966e810c61 admin admin] The token cache is being invalidate because project 054c2935d14045639844e73344ea71d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:39:36.423368 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d96f15a7-35d5-4ee6-aed2-20966e810c61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:36.424237 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d96f15a7-35d5-4ee6-aed2-20966e810c61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:36.426351 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1201/2376] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:39:36 2022] DELETE /identity/v3/projects/054c2935d14045639844e73344ea71d9 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:39:36.431192 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45fb17-60d7-4d40-9a07-a47608943df8 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:36.447035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45fb17-60d7-4d40-9a07-a47608943df8 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:36.503037 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e6e22-5a16-4dcd-9d3a-1e581288624d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:36.503718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e6e22-5a16-4dcd-9d3a-1e581288624d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:36.503997 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e6e22-5a16-4dcd-9d3a-1e581288624d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:36.504267 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e6e22-5a16-4dcd-9d3a-1e581288624d admin admin] PATH_INFO: `/v3/users/388beff30a9f4a3997fddc518ddd910c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:36.505363 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e6e22-5a16-4dcd-9d3a-1e581288624d admin admin] RBAC: Authorizing `identity:delete_user(user_id=388beff30a9f4a3997fddc518ddd910c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:36.507335 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e6e22-5a16-4dcd-9d3a-1e581288624d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:36.554898 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f19e6e22-5a16-4dcd-9d3a-1e581288624d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '388beff30a9f4a3997fddc518ddd910c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:36.561752 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f19e6e22-5a16-4dcd-9d3a-1e581288624d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '388beff30a9f4a3997fddc518ddd910c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:36.562460 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f19e6e22-5a16-4dcd-9d3a-1e581288624d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '388beff30a9f4a3997fddc518ddd910c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:36.569700 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f19e6e22-5a16-4dcd-9d3a-1e581288624d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '388beff30a9f4a3997fddc518ddd910c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:36.582308 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f19e6e22-5a16-4dcd-9d3a-1e581288624d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '388beff30a9f4a3997fddc518ddd910c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:36.589659 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1176/2377] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:39:36 2022] DELETE /identity/v3/users/388beff30a9f4a3997fddc518ddd910c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:39:36.898462 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f15a7-35d5-4ee6-aed2-20966e810c61 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:36.900592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d96f15a7-35d5-4ee6-aed2-20966e810c61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:36.920755 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f15a7-35d5-4ee6-aed2-20966e810c61 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:36.930806 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:37.006777 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce39eb-bdbb-4c78-812b-3a9ab8ff05b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:37.007458 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce39eb-bdbb-4c78-812b-3a9ab8ff05b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:37.007741 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce39eb-bdbb-4c78-812b-3a9ab8ff05b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:37.009194 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce39eb-bdbb-4c78-812b-3a9ab8ff05b5 admin admin] PATH_INFO: `/v3/projects/cb8039e1d29a491eb68f8df1e81c51f8` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:37.009862 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce39eb-bdbb-4c78-812b-3a9ab8ff05b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb8039e1d29a491eb68f8df1e81c51f8)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:37.012004 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce39eb-bdbb-4c78-812b-3a9ab8ff05b5 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:37.083441 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-57ce39eb-bdbb-4c78-812b-3a9ab8ff05b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb8039e1d29a491eb68f8df1e81c51f8', 'request_id': 'req-57ce39eb-bdbb-4c78-812b-3a9ab8ff05b5'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:37.090464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-57ce39eb-bdbb-4c78-812b-3a9ab8ff05b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb8039e1d29a491eb68f8df1e81c51f8', 'request_id': 'req-57ce39eb-bdbb-4c78-812b-3a9ab8ff05b5'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:37.096309 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-57ce39eb-bdbb-4c78-812b-3a9ab8ff05b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb8039e1d29a491eb68f8df1e81c51f8', 'request_id': 'req-57ce39eb-bdbb-4c78-812b-3a9ab8ff05b5'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:37.108597 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-57ce39eb-bdbb-4c78-812b-3a9ab8ff05b5 admin admin] The token cache is being invalidate because project cb8039e1d29a491eb68f8df1e81c51f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:39:37.108889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-57ce39eb-bdbb-4c78-812b-3a9ab8ff05b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:37.110091 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-57ce39eb-bdbb-4c78-812b-3a9ab8ff05b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:37.111514 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1202/2378] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:39:36 2022] DELETE /identity/v3/projects/cb8039e1d29a491eb68f8df1e81c51f8 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:39:37.116113 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d93c8a-ca26-4079-a1e6-fb8c740f190c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:37.116587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d93c8a-ca26-4079-a1e6-fb8c740f190c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:37.116927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d93c8a-ca26-4079-a1e6-fb8c740f190c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:37.137956 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d4d93c8a-ca26-4079-a1e6-fb8c740f190c None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:37.139281 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d4d93c8a-ca26-4079-a1e6-fb8c740f190c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:37.186074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:37.207154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1177/2379] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:37.215007 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca623b6-5b25-4532-bc7b-4065bd4b9cec None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:37.215298 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca623b6-5b25-4532-bc7b-4065bd4b9cec None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:37.215576 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca623b6-5b25-4532-bc7b-4065bd4b9cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:37.235745 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-6ca623b6-5b25-4532-bc7b-4065bd4b9cec None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:37.236975 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6ca623b6-5b25-4532-bc7b-4065bd4b9cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:37.246901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1203/2380] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:37.251076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d93c8a-ca26-4079-a1e6-fb8c740f190c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:37.263272 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d93c8a-ca26-4079-a1e6-fb8c740f190c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:37.271246 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d1dbc3-90da-4dc7-89cf-6c2d01a19213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:37.272016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1dbc3-90da-4dc7-89cf-6c2d01a19213 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:37.272354 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1dbc3-90da-4dc7-89cf-6c2d01a19213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:37.272654 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1dbc3-90da-4dc7-89cf-6c2d01a19213 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:37.273440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d1dbc3-90da-4dc7-89cf-6c2d01a19213 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:39:37.273774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d1dbc3-90da-4dc7-89cf-6c2d01a19213 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:37.276257 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d1dbc3-90da-4dc7-89cf-6c2d01a19213 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:37.291891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1178/2381] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:39:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:39:37.296600 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca623b6-5b25-4532-bc7b-4065bd4b9cec None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:37.307263 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca623b6-5b25-4532-bc7b-4065bd4b9cec None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:37.312069 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ba3b8-6698-4c18-be15-0f98a5f40821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:37.312702 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ba3b8-6698-4c18-be15-0f98a5f40821 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:37.312978 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ba3b8-6698-4c18-be15-0f98a5f40821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:37.313245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ba3b8-6698-4c18-be15-0f98a5f40821 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:37.313872 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891ba3b8-6698-4c18-be15-0f98a5f40821 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:37.315961 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891ba3b8-6698-4c18-be15-0f98a5f40821 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:37.346715 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1204/2382] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:39:37 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:39:37.351078 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d1dbc3-90da-4dc7-89cf-6c2d01a19213 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:37.361265 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d1dbc3-90da-4dc7-89cf-6c2d01a19213 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:37.424471 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfe0ca-c2f5-4a58-a0d2-edf998d03252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:37.425105 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfe0ca-c2f5-4a58-a0d2-edf998d03252 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:37.425451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfe0ca-c2f5-4a58-a0d2-edf998d03252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:37.425792 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfe0ca-c2f5-4a58-a0d2-edf998d03252 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:37.427161 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bfe0ca-c2f5-4a58-a0d2-edf998d03252 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:37.428739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bfe0ca-c2f5-4a58-a0d2-edf998d03252 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:37.454248 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1179/2383] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:39:37 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:39:37.455404 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ba3b8-6698-4c18-be15-0f98a5f40821 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:37.467597 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ba3b8-6698-4c18-be15-0f98a5f40821 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:37.472370 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22204be2-4d95-44bd-9005-1f40af1235f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:37.473050 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22204be2-4d95-44bd-9005-1f40af1235f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:37.473396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22204be2-4d95-44bd-9005-1f40af1235f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:37.473678 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22204be2-4d95-44bd-9005-1f40af1235f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:37.474370 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22204be2-4d95-44bd-9005-1f40af1235f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:37.475972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22204be2-4d95-44bd-9005-1f40af1235f1 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:37.486308 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1205/2384] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:39:37.490279 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfe0ca-c2f5-4a58-a0d2-edf998d03252 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:37.500737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfe0ca-c2f5-4a58-a0d2-edf998d03252 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:37.505332 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584ed39-372f-4be9-b67e-e3c57fd74f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:37.505956 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584ed39-372f-4be9-b67e-e3c57fd74f89 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:37.506280 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584ed39-372f-4be9-b67e-e3c57fd74f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:37.506540 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584ed39-372f-4be9-b67e-e3c57fd74f89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:37.507209 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3584ed39-372f-4be9-b67e-e3c57fd74f89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:37.509337 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3584ed39-372f-4be9-b67e-e3c57fd74f89 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:37.520446 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1180/2385] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:39:37.525485 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22204be2-4d95-44bd-9005-1f40af1235f1 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:37.535538 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22204be2-4d95-44bd-9005-1f40af1235f1 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:37.541040 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c023ad-b550-45ef-adb3-26290a2e79d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:37.541691 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c023ad-b550-45ef-adb3-26290a2e79d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:37.541971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c023ad-b550-45ef-adb3-26290a2e79d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:37.542278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c023ad-b550-45ef-adb3-26290a2e79d1 admin admin] PATH_INFO: `/v3/projects/5197edf9e43342a3a913ee989ce8fd6e/users/bcb2ca22f9894f1ebbb4af277b59ab47/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:37.557041 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c023ad-b550-45ef-adb3-26290a2e79d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5197edf9e43342a3a913ee989ce8fd6e, user_id=bcb2ca22f9894f1ebbb4af277b59ab47, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:37.559813 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c023ad-b550-45ef-adb3-26290a2e79d1 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:37.573952 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1206/2386] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:39:37 2022] PUT /identity/v3/projects/5197edf9e43342a3a913ee989ce8fd6e/users/bcb2ca22f9894f1ebbb4af277b59ab47/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:39:37.585224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd6199-9b5e-472d-91e2-568419252382 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:37.585505 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd6199-9b5e-472d-91e2-568419252382 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:37.585795 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd6199-9b5e-472d-91e2-568419252382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:37.616210 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a3dd6199-9b5e-472d-91e2-568419252382 None None] MFA Rules not processed for user `bcb2ca22f9894f1ebbb4af277b59ab47`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:37.617430 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a3dd6199-9b5e-472d-91e2-568419252382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:37.684225 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1181/2387] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:39:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:37.688318 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c023ad-b550-45ef-adb3-26290a2e79d1 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:37.730286 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:37.762942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c023ad-b550-45ef-adb3-26290a2e79d1 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:37.767552 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3909e3b-562d-4f64-8f06-5bc8d6a8b9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:37.768247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3909e3b-562d-4f64-8f06-5bc8d6a8b9c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:37.768533 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3909e3b-562d-4f64-8f06-5bc8d6a8b9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:37.768809 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3909e3b-562d-4f64-8f06-5bc8d6a8b9c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:37.769453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3909e3b-562d-4f64-8f06-5bc8d6a8b9c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:37.771656 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3909e3b-562d-4f64-8f06-5bc8d6a8b9c3 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:37.800011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1207/2388] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:39:37 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:39:37.804096 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd6199-9b5e-472d-91e2-568419252382 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:37.816211 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd6199-9b5e-472d-91e2-568419252382 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:37.873974 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:37.891622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe5758-d51c-4bd8-86a1-bb8328e572c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:37.892238 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe5758-d51c-4bd8-86a1-bb8328e572c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:37.892548 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe5758-d51c-4bd8-86a1-bb8328e572c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:37.892820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe5758-d51c-4bd8-86a1-bb8328e572c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:37.893449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe5758-d51c-4bd8-86a1-bb8328e572c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:37.895415 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe5758-d51c-4bd8-86a1-bb8328e572c5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:37.916973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1182/2389] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:39:37 2022] POST /identity/v3/users => generated 299 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:39:37.921022 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3909e3b-562d-4f64-8f06-5bc8d6a8b9c3 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:37.931343 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3909e3b-562d-4f64-8f06-5bc8d6a8b9c3 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:37.936047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f231b8-c355-44e5-8de6-e5119634c42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:37.936655 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f231b8-c355-44e5-8de6-e5119634c42f admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:37.936939 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f231b8-c355-44e5-8de6-e5119634c42f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:37.937214 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f231b8-c355-44e5-8de6-e5119634c42f admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:37.937862 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f231b8-c355-44e5-8de6-e5119634c42f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:37.939497 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f231b8-c355-44e5-8de6-e5119634c42f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:37.950262 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1208/2390] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:39:37.954455 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe5758-d51c-4bd8-86a1-bb8328e572c5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:37.965244 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe5758-d51c-4bd8-86a1-bb8328e572c5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:37.969862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d84a2-4c49-4af0-bea1-66b0bc54ce97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:37.970545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d84a2-4c49-4af0-bea1-66b0bc54ce97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:37.970831 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d84a2-4c49-4af0-bea1-66b0bc54ce97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:37.971109 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d84a2-4c49-4af0-bea1-66b0bc54ce97 admin admin] PATH_INFO: `/v3/projects/111c0afd9ad04b9484463c7964a276b8/users/3c662f2fa027420d94f1bdbd46f0bfe6/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:37.986601 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0d84a2-4c49-4af0-bea1-66b0bc54ce97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=111c0afd9ad04b9484463c7964a276b8, user_id=3c662f2fa027420d94f1bdbd46f0bfe6, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:37.989482 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0d84a2-4c49-4af0-bea1-66b0bc54ce97 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:38.003586 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1183/2391] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:39:37 2022] PUT /identity/v3/projects/111c0afd9ad04b9484463c7964a276b8/users/3c662f2fa027420d94f1bdbd46f0bfe6/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:39:38.007524 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f231b8-c355-44e5-8de6-e5119634c42f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:38.018041 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f231b8-c355-44e5-8de6-e5119634c42f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:38.079493 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda1ea3-3a47-4ce7-b2ad-3d549c6a4c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:38.080101 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda1ea3-3a47-4ce7-b2ad-3d549c6a4c65 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:38.080379 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda1ea3-3a47-4ce7-b2ad-3d549c6a4c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:38.080648 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda1ea3-3a47-4ce7-b2ad-3d549c6a4c65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:38.081292 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda1ea3-3a47-4ce7-b2ad-3d549c6a4c65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:38.083301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda1ea3-3a47-4ce7-b2ad-3d549c6a4c65 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:38.097897 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1209/2392] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:39:38 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:39:38.099316 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d84a2-4c49-4af0-bea1-66b0bc54ce97 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:38.111499 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d84a2-4c49-4af0-bea1-66b0bc54ce97 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:38.116326 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5225945-420b-426d-adef-5e047bc1d41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:38.116989 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5225945-420b-426d-adef-5e047bc1d41a admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:38.117270 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5225945-420b-426d-adef-5e047bc1d41a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:38.117587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5225945-420b-426d-adef-5e047bc1d41a admin admin] PATH_INFO: `/v3/projects/111c0afd9ad04b9484463c7964a276b8/users/3c662f2fa027420d94f1bdbd46f0bfe6/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:38.120058 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5225945-420b-426d-adef-5e047bc1d41a admin admin] RBAC: Authorizing `identity:create_grant(project_id=111c0afd9ad04b9484463c7964a276b8, user_id=3c662f2fa027420d94f1bdbd46f0bfe6, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:38.122730 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5225945-420b-426d-adef-5e047bc1d41a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:38.133758 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1184/2393] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:39:38 2022] PUT /identity/v3/projects/111c0afd9ad04b9484463c7964a276b8/users/3c662f2fa027420d94f1bdbd46f0bfe6/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:39:38.144200 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577603f-8aef-4597-be46-c587bf2d9436 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:38.144488 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577603f-8aef-4597-be46-c587bf2d9436 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:38.144767 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577603f-8aef-4597-be46-c587bf2d9436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:38.176713 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-9577603f-8aef-4597-be46-c587bf2d9436 None None] MFA Rules not processed for user `3c662f2fa027420d94f1bdbd46f0bfe6`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:38.179069 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9577603f-8aef-4597-be46-c587bf2d9436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:38.266885 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1210/2394] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:39:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:39:38.278117 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5225945-420b-426d-adef-5e047bc1d41a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:38.279843 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b5225945-420b-426d-adef-5e047bc1d41a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:38.365397 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5225945-420b-426d-adef-5e047bc1d41a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:38.369200 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29149d6a-26e6-436d-b6cc-c4f4193d5181 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:38.369819 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29149d6a-26e6-436d-b6cc-c4f4193d5181 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:38.370101 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29149d6a-26e6-436d-b6cc-c4f4193d5181 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:38.370400 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29149d6a-26e6-436d-b6cc-c4f4193d5181 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:38.380390 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29149d6a-26e6-436d-b6cc-c4f4193d5181 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:38.381840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29149d6a-26e6-436d-b6cc-c4f4193d5181 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:38.393404 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1185/2395] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:38 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:39:38.423383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577603f-8aef-4597-be46-c587bf2d9436 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:38.456493 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:38.492908 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577603f-8aef-4597-be46-c587bf2d9436 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:38.497497 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800fa215-828f-4101-8b42-aa57ef94992f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:38.498133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800fa215-828f-4101-8b42-aa57ef94992f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:38.498438 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800fa215-828f-4101-8b42-aa57ef94992f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:38.498704 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800fa215-828f-4101-8b42-aa57ef94992f admin admin] PATH_INFO: `/v3/users/bcb2ca22f9894f1ebbb4af277b59ab47` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:38.499854 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800fa215-828f-4101-8b42-aa57ef94992f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcb2ca22f9894f1ebbb4af277b59ab47)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:38.501771 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800fa215-828f-4101-8b42-aa57ef94992f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:38.552341 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-800fa215-828f-4101-8b42-aa57ef94992f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcb2ca22f9894f1ebbb4af277b59ab47'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:38.563465 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-800fa215-828f-4101-8b42-aa57ef94992f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcb2ca22f9894f1ebbb4af277b59ab47'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:38.564213 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-800fa215-828f-4101-8b42-aa57ef94992f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcb2ca22f9894f1ebbb4af277b59ab47'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:38.564827 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-800fa215-828f-4101-8b42-aa57ef94992f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcb2ca22f9894f1ebbb4af277b59ab47'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:38.583393 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-800fa215-828f-4101-8b42-aa57ef94992f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcb2ca22f9894f1ebbb4af277b59ab47'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:38.596565 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1211/2396] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:39:38 2022] DELETE /identity/v3/users/bcb2ca22f9894f1ebbb4af277b59ab47 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:39:38.601248 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12441b25-ef87-428d-b5d4-1ffc352b82cc None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:38.601537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12441b25-ef87-428d-b5d4-1ffc352b82cc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:38.601997 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12441b25-ef87-428d-b5d4-1ffc352b82cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:38.622019 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-12441b25-ef87-428d-b5d4-1ffc352b82cc None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:38.623145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-12441b25-ef87-428d-b5d4-1ffc352b82cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:38.681988 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1186/2397] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:39:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:38.697211 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800fa215-828f-4101-8b42-aa57ef94992f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:38.698923 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-800fa215-828f-4101-8b42-aa57ef94992f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:38.754856 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800fa215-828f-4101-8b42-aa57ef94992f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:38.759068 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b70ca-60e7-44b1-b73d-32f791528fcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:38.759660 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b70ca-60e7-44b1-b73d-32f791528fcf service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:38.759937 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b70ca-60e7-44b1-b73d-32f791528fcf service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:38.760207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b70ca-60e7-44b1-b73d-32f791528fcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:38.768913 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5b70ca-60e7-44b1-b73d-32f791528fcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:38.770355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5b70ca-60e7-44b1-b73d-32f791528fcf service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:38.779747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1212/2398] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:39:39.049666 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12441b25-ef87-428d-b5d4-1ffc352b82cc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:39.050652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-12441b25-ef87-428d-b5d4-1ffc352b82cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:39.063198 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12441b25-ef87-428d-b5d4-1ffc352b82cc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:39.068152 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de91caa-430e-426e-9762-70f91635288a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:39.068793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de91caa-430e-426e-9762-70f91635288a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:39.069076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de91caa-430e-426e-9762-70f91635288a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:39.069351 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de91caa-430e-426e-9762-70f91635288a admin admin] PATH_INFO: `/v3/projects/5197edf9e43342a3a913ee989ce8fd6e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:39.070533 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de91caa-430e-426e-9762-70f91635288a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5197edf9e43342a3a913ee989ce8fd6e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:39.072451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de91caa-430e-426e-9762-70f91635288a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:39.084804 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:39.120532 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9de91caa-430e-426e-9762-70f91635288a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5197edf9e43342a3a913ee989ce8fd6e', 'request_id': 'req-9de91caa-430e-426e-9762-70f91635288a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:39.126930 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9de91caa-430e-426e-9762-70f91635288a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5197edf9e43342a3a913ee989ce8fd6e', 'request_id': 'req-9de91caa-430e-426e-9762-70f91635288a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:39.134624 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9de91caa-430e-426e-9762-70f91635288a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5197edf9e43342a3a913ee989ce8fd6e', 'request_id': 'req-9de91caa-430e-426e-9762-70f91635288a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:39.145057 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9de91caa-430e-426e-9762-70f91635288a admin admin] The token cache is being invalidate because project 5197edf9e43342a3a913ee989ce8fd6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:39:39.145347 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9de91caa-430e-426e-9762-70f91635288a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:39.146264 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9de91caa-430e-426e-9762-70f91635288a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:39.147791 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1187/2399] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:39:39 2022] DELETE /identity/v3/projects/5197edf9e43342a3a913ee989ce8fd6e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:39:39.152548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b70ca-60e7-44b1-b73d-32f791528fcf service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:39.162127 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b70ca-60e7-44b1-b73d-32f791528fcf service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:39.209582 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:39.221267 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12c78a-9917-4935-b948-9bb26ee9aa1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:39.221886 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12c78a-9917-4935-b948-9bb26ee9aa1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:39.222511 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12c78a-9917-4935-b948-9bb26ee9aa1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:39.222511 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12c78a-9917-4935-b948-9bb26ee9aa1a admin admin] PATH_INFO: `/v3/users/3c662f2fa027420d94f1bdbd46f0bfe6` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:39.223569 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e12c78a-9917-4935-b948-9bb26ee9aa1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c662f2fa027420d94f1bdbd46f0bfe6)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:39.225491 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e12c78a-9917-4935-b948-9bb26ee9aa1a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:39.276335 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8e12c78a-9917-4935-b948-9bb26ee9aa1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c662f2fa027420d94f1bdbd46f0bfe6'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:39.283732 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8e12c78a-9917-4935-b948-9bb26ee9aa1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c662f2fa027420d94f1bdbd46f0bfe6'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:39.284305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8e12c78a-9917-4935-b948-9bb26ee9aa1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c662f2fa027420d94f1bdbd46f0bfe6'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:39.284913 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8e12c78a-9917-4935-b948-9bb26ee9aa1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c662f2fa027420d94f1bdbd46f0bfe6'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:39.297934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8e12c78a-9917-4935-b948-9bb26ee9aa1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c662f2fa027420d94f1bdbd46f0bfe6'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:39.311748 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1213/2400] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:39:39 2022] DELETE /identity/v3/users/3c662f2fa027420d94f1bdbd46f0bfe6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:39:39.584777 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de91caa-430e-426e-9762-70f91635288a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:39.586497 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9de91caa-430e-426e-9762-70f91635288a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:39.600006 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de91caa-430e-426e-9762-70f91635288a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:39.660111 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3945ef2-bd63-4682-8b44-673285412f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:39.660792 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3945ef2-bd63-4682-8b44-673285412f86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:39.661021 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3945ef2-bd63-4682-8b44-673285412f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:39.661296 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3945ef2-bd63-4682-8b44-673285412f86 admin admin] PATH_INFO: `/v3/projects/111c0afd9ad04b9484463c7964a276b8` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:39.662430 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3945ef2-bd63-4682-8b44-673285412f86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=111c0afd9ad04b9484463c7964a276b8)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:39.664355 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3945ef2-bd63-4682-8b44-673285412f86 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:39.710288 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3945ef2-bd63-4682-8b44-673285412f86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '111c0afd9ad04b9484463c7964a276b8', 'request_id': 'req-a3945ef2-bd63-4682-8b44-673285412f86'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:39.716892 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3945ef2-bd63-4682-8b44-673285412f86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '111c0afd9ad04b9484463c7964a276b8', 'request_id': 'req-a3945ef2-bd63-4682-8b44-673285412f86'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:39.724162 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3945ef2-bd63-4682-8b44-673285412f86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '111c0afd9ad04b9484463c7964a276b8', 'request_id': 'req-a3945ef2-bd63-4682-8b44-673285412f86'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:39.734485 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3945ef2-bd63-4682-8b44-673285412f86 admin admin] The token cache is being invalidate because project 111c0afd9ad04b9484463c7964a276b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:39:39.734768 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3945ef2-bd63-4682-8b44-673285412f86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:39.735399 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3945ef2-bd63-4682-8b44-673285412f86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:39.736854 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1188/2401] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:39:39 2022] DELETE /identity/v3/projects/111c0afd9ad04b9484463c7964a276b8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:39:39.741968 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae3af7-554e-457d-be6b-7091240392f7 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:39.742296 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae3af7-554e-457d-be6b-7091240392f7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:39.742579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae3af7-554e-457d-be6b-7091240392f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:39.762844 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2eae3af7-554e-457d-be6b-7091240392f7 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:39.764047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2eae3af7-554e-457d-be6b-7091240392f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:39.824468 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1214/2402] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:39.832175 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e18f1-0e0b-4191-8ab5-b3109f6b99c8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:39.832458 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e18f1-0e0b-4191-8ab5-b3109f6b99c8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:39.832728 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e18f1-0e0b-4191-8ab5-b3109f6b99c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:39.852732 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f78e18f1-0e0b-4191-8ab5-b3109f6b99c8 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:39.853939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f78e18f1-0e0b-4191-8ab5-b3109f6b99c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:39.862862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1189/2403] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:39.867152 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae3af7-554e-457d-be6b-7091240392f7 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:39.877590 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae3af7-554e-457d-be6b-7091240392f7 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:39.882021 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f9cda-cdb1-4d62-b932-39a182cee400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:39.882653 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f9cda-cdb1-4d62-b932-39a182cee400 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:39.882946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f9cda-cdb1-4d62-b932-39a182cee400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:39.883224 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f9cda-cdb1-4d62-b932-39a182cee400 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:39.883872 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f9cda-cdb1-4d62-b932-39a182cee400 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:39:39.884182 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f9cda-cdb1-4d62-b932-39a182cee400 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:39.885773 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f9cda-cdb1-4d62-b932-39a182cee400 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:39.897526 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1215/2404] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:39:39.902070 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e18f1-0e0b-4191-8ab5-b3109f6b99c8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:39.911448 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e18f1-0e0b-4191-8ab5-b3109f6b99c8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:39.916498 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63e844-a91b-4362-a736-0ebfb7241a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:39.917087 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63e844-a91b-4362-a736-0ebfb7241a83 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:39.917366 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63e844-a91b-4362-a736-0ebfb7241a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:39.917638 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63e844-a91b-4362-a736-0ebfb7241a83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:39.918338 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63e844-a91b-4362-a736-0ebfb7241a83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:39.920221 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63e844-a91b-4362-a736-0ebfb7241a83 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:39.947466 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1190/2405] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:39:39 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:39:39.951555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f9cda-cdb1-4d62-b932-39a182cee400 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:39.961935 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f9cda-cdb1-4d62-b932-39a182cee400 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:40.012418 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:40.019445 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b080e8c-b052-4e83-8287-d0b5eb551f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:40.020115 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b080e8c-b052-4e83-8287-d0b5eb551f1c admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:40.020851 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b080e8c-b052-4e83-8287-d0b5eb551f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:40.021160 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b080e8c-b052-4e83-8287-d0b5eb551f1c admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:40.022040 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b080e8c-b052-4e83-8287-d0b5eb551f1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:40.024810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b080e8c-b052-4e83-8287-d0b5eb551f1c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:40.049772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1216/2406] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:39:39 2022] POST /identity/v3/users => generated 285 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:39:40.053851 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63e844-a91b-4362-a736-0ebfb7241a83 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:40.060750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:40.067224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63e844-a91b-4362-a736-0ebfb7241a83 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:40.073489 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd87c98-9e88-43c6-8c9d-7ea1954b277b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:40.074098 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd87c98-9e88-43c6-8c9d-7ea1954b277b admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:40.074414 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd87c98-9e88-43c6-8c9d-7ea1954b277b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:40.074714 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd87c98-9e88-43c6-8c9d-7ea1954b277b admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:40.075361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd87c98-9e88-43c6-8c9d-7ea1954b277b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:40.076952 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd87c98-9e88-43c6-8c9d-7ea1954b277b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:40.087368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1191/2407] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:39:40.091589 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b080e8c-b052-4e83-8287-d0b5eb551f1c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:40.101451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b080e8c-b052-4e83-8287-d0b5eb551f1c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:40.107071 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50ad79-8ba3-4bd4-86b2-f25911cb590c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:40.107657 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50ad79-8ba3-4bd4-86b2-f25911cb590c admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:40.107967 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50ad79-8ba3-4bd4-86b2-f25911cb590c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:40.108241 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50ad79-8ba3-4bd4-86b2-f25911cb590c admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:40.108923 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd50ad79-8ba3-4bd4-86b2-f25911cb590c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:40.110564 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd50ad79-8ba3-4bd4-86b2-f25911cb590c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:40.120931 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1217/2408] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:39:40.125025 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd87c98-9e88-43c6-8c9d-7ea1954b277b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:40.134993 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd87c98-9e88-43c6-8c9d-7ea1954b277b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:40.139747 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06acc2-99c3-406c-974c-5bdbf164ecad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:40.140391 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06acc2-99c3-406c-974c-5bdbf164ecad admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:40.140672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06acc2-99c3-406c-974c-5bdbf164ecad admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:40.140962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06acc2-99c3-406c-974c-5bdbf164ecad admin admin] PATH_INFO: `/v3/projects/4882800e4d0c4aae858353231627b159/users/dcd1f9d18d8e434e8a2e9740ee767b54/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:40.155945 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06acc2-99c3-406c-974c-5bdbf164ecad admin admin] RBAC: Authorizing `identity:create_grant(project_id=4882800e4d0c4aae858353231627b159, user_id=dcd1f9d18d8e434e8a2e9740ee767b54, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:40.158627 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06acc2-99c3-406c-974c-5bdbf164ecad admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:40.171669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1192/2409] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:39:40 2022] PUT /identity/v3/projects/4882800e4d0c4aae858353231627b159/users/dcd1f9d18d8e434e8a2e9740ee767b54/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:39:40.237079 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e4fa5-32cd-47f3-9a9a-b17db684f939 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:40.237617 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e4fa5-32cd-47f3-9a9a-b17db684f939 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:40.237617 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e4fa5-32cd-47f3-9a9a-b17db684f939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:40.265824 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5e1e4fa5-32cd-47f3-9a9a-b17db684f939 None None] MFA Rules not processed for user `dcd1f9d18d8e434e8a2e9740ee767b54`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:40.267047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5e1e4fa5-32cd-47f3-9a9a-b17db684f939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:40.337045 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1218/2410] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:39:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:40.348227 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06acc2-99c3-406c-974c-5bdbf164ecad admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:40.349868 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0a06acc2-99c3-406c-974c-5bdbf164ecad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:40.407220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06acc2-99c3-406c-974c-5bdbf164ecad admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:40.410830 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8333d0-fdbe-4632-9ad7-211b280fc4e9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2c872129414105a112661cc2b477e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:40.411427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8333d0-fdbe-4632-9ad7-211b280fc4e9 service glance] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:40.411708 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8333d0-fdbe-4632-9ad7-211b280fc4e9 service glance] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:40.411978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8333d0-fdbe-4632-9ad7-211b280fc4e9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:40.421211 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8333d0-fdbe-4632-9ad7-211b280fc4e9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:40.422771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8333d0-fdbe-4632-9ad7-211b280fc4e9 service glance] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:40.432089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1193/2411] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:40 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:39:50.334086 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=40,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:50.427555 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=40,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:53.248570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e4fa5-32cd-47f3-9a9a-b17db684f939 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:53.250014 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5e1e4fa5-32cd-47f3-9a9a-b17db684f939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:53.341528 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e4fa5-32cd-47f3-9a9a-b17db684f939 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:53.346560 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f73d3-8d51-4cca-89c8-995d8310f366 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12302cdb2854b0684179647a6dadf6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:53.347163 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f73d3-8d51-4cca-89c8-995d8310f366 service placement] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:53.347447 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f73d3-8d51-4cca-89c8-995d8310f366 service placement] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:53.347720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f73d3-8d51-4cca-89c8-995d8310f366 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:53.349557 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-928f73d3-8d51-4cca-89c8-995d8310f366 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:53.361416 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f73d3-8d51-4cca-89c8-995d8310f366 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:53.363021 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f73d3-8d51-4cca-89c8-995d8310f366 service placement] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:53.373708 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1219/2412] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:40:03.369235 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=41 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:16.234561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c7820-1109-4395-9c0f-7cacd85a8530 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:16.235103 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c7820-1109-4395-9c0f-7cacd85a8530 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:16.235103 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c7820-1109-4395-9c0f-7cacd85a8530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:16.255032 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d50c7820-1109-4395-9c0f-7cacd85a8530 None None] MFA Rules not processed for user `024adeb8a8e14d678ae14174d9555152`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:16.256251 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d50c7820-1109-4395-9c0f-7cacd85a8530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:16.313110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1194/2413] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:40:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:25.113548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f73d3-8d51-4cca-89c8-995d8310f366 service placement] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:25.200529 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f73d3-8d51-4cca-89c8-995d8310f366 service placement] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:25.207372 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80352994-b396-49b4-87f8-6187bae33786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:25.208333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80352994-b396-49b4-87f8-6187bae33786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:25.209285 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80352994-b396-49b4-87f8-6187bae33786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:25.209285 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80352994-b396-49b4-87f8-6187bae33786 admin admin] PATH_INFO: `/v3/users/dcd1f9d18d8e434e8a2e9740ee767b54` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:25.210792 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80352994-b396-49b4-87f8-6187bae33786 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcd1f9d18d8e434e8a2e9740ee767b54)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:25.212786 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80352994-b396-49b4-87f8-6187bae33786 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:25.272502 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-80352994-b396-49b4-87f8-6187bae33786 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcd1f9d18d8e434e8a2e9740ee767b54'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:25.281236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-80352994-b396-49b4-87f8-6187bae33786 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcd1f9d18d8e434e8a2e9740ee767b54'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:25.282067 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-80352994-b396-49b4-87f8-6187bae33786 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcd1f9d18d8e434e8a2e9740ee767b54'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:25.282779 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-80352994-b396-49b4-87f8-6187bae33786 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcd1f9d18d8e434e8a2e9740ee767b54'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:25.295432 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-80352994-b396-49b4-87f8-6187bae33786 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcd1f9d18d8e434e8a2e9740ee767b54'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:25.310159 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1220/2414] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:40:25 2022] DELETE /identity/v3/users/dcd1f9d18d8e434e8a2e9740ee767b54 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:25.314623 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf17533-b2cb-420e-bd11-a114e4d1198f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:25.315110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf17533-b2cb-420e-bd11-a114e4d1198f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:25.315346 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf17533-b2cb-420e-bd11-a114e4d1198f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:25.335326 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1bf17533-b2cb-420e-bd11-a114e4d1198f None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:25.336572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1bf17533-b2cb-420e-bd11-a114e4d1198f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:25.397441 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1195/2415] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:25.407206 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80352994-b396-49b4-87f8-6187bae33786 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:25.408644 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-80352994-b396-49b4-87f8-6187bae33786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:25.468099 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80352994-b396-49b4-87f8-6187bae33786 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:25.471687 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e697066-2242-4abf-8e8c-5928501a0850 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:25.472292 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e697066-2242-4abf-8e8c-5928501a0850 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:25.472571 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e697066-2242-4abf-8e8c-5928501a0850 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:25.472868 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e697066-2242-4abf-8e8c-5928501a0850 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:25.482414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e697066-2242-4abf-8e8c-5928501a0850 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:25.483833 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e697066-2242-4abf-8e8c-5928501a0850 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:25.493859 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1221/2416] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:40:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:40:25.770957 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf17533-b2cb-420e-bd11-a114e4d1198f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:25.772504 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1bf17533-b2cb-420e-bd11-a114e4d1198f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:25.784803 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf17533-b2cb-420e-bd11-a114e4d1198f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:25.790252 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb0a8b-ec16-4885-a28e-822e5231133f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:25.790976 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb0a8b-ec16-4885-a28e-822e5231133f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:25.791340 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb0a8b-ec16-4885-a28e-822e5231133f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:25.791684 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb0a8b-ec16-4885-a28e-822e5231133f admin admin] PATH_INFO: `/v3/projects/4882800e4d0c4aae858353231627b159` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:25.792890 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb0a8b-ec16-4885-a28e-822e5231133f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4882800e4d0c4aae858353231627b159)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:25.794873 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb0a8b-ec16-4885-a28e-822e5231133f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:25.856270 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-86bb0a8b-ec16-4885-a28e-822e5231133f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4882800e4d0c4aae858353231627b159', 'request_id': 'req-86bb0a8b-ec16-4885-a28e-822e5231133f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:25.863452 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-86bb0a8b-ec16-4885-a28e-822e5231133f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4882800e4d0c4aae858353231627b159', 'request_id': 'req-86bb0a8b-ec16-4885-a28e-822e5231133f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:25.872329 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-86bb0a8b-ec16-4885-a28e-822e5231133f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4882800e4d0c4aae858353231627b159', 'request_id': 'req-86bb0a8b-ec16-4885-a28e-822e5231133f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:25.882946 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-86bb0a8b-ec16-4885-a28e-822e5231133f admin admin] The token cache is being invalidate because project 4882800e4d0c4aae858353231627b159 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:40:25.883234 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-86bb0a8b-ec16-4885-a28e-822e5231133f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:25.883856 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-86bb0a8b-ec16-4885-a28e-822e5231133f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:25.885546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1196/2417] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:40:25 2022] DELETE /identity/v3/projects/4882800e4d0c4aae858353231627b159 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:40:25.890400 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f30e9b-4b87-4465-acf7-67a3f7522f6f None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:25.890632 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f30e9b-4b87-4465-acf7-67a3f7522f6f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:25.890937 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f30e9b-4b87-4465-acf7-67a3f7522f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:25.912116 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-18f30e9b-4b87-4465-acf7-67a3f7522f6f None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:25.914132 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-18f30e9b-4b87-4465-acf7-67a3f7522f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:25.933282 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:25.976435 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1222/2418] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:25.984169 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424df819-b0df-4524-a189-e9d64ffbadd9 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:25.984485 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424df819-b0df-4524-a189-e9d64ffbadd9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:25.984766 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424df819-b0df-4524-a189-e9d64ffbadd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:26.005022 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-424df819-b0df-4524-a189-e9d64ffbadd9 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:26.006244 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-424df819-b0df-4524-a189-e9d64ffbadd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:26.014245 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1197/2419] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:26.018359 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f30e9b-4b87-4465-acf7-67a3f7522f6f None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:26.028268 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f30e9b-4b87-4465-acf7-67a3f7522f6f None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:26.032786 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d6524-94df-45e5-b739-5c69712dbc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:26.033380 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d6524-94df-45e5-b739-5c69712dbc60 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:26.033691 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d6524-94df-45e5-b739-5c69712dbc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:26.033926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d6524-94df-45e5-b739-5c69712dbc60 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:26.034607 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d6524-94df-45e5-b739-5c69712dbc60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:40:26.034908 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d6524-94df-45e5-b739-5c69712dbc60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:26.036491 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d6524-94df-45e5-b739-5c69712dbc60 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:26.050507 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1223/2420] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:40:26.055556 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424df819-b0df-4524-a189-e9d64ffbadd9 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:26.065112 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424df819-b0df-4524-a189-e9d64ffbadd9 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:26.070105 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33703093-9dc8-4b87-94c1-51253c36542c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:26.070755 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33703093-9dc8-4b87-94c1-51253c36542c admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:26.071037 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33703093-9dc8-4b87-94c1-51253c36542c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:26.071311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33703093-9dc8-4b87-94c1-51253c36542c admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:26.071990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33703093-9dc8-4b87-94c1-51253c36542c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:26.073896 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33703093-9dc8-4b87-94c1-51253c36542c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:26.096941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:26.107944 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1198/2421] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:40:26 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:40:26.109198 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d6524-94df-45e5-b739-5c69712dbc60 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:26.119601 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d6524-94df-45e5-b739-5c69712dbc60 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:26.176487 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e70c00-0b0e-458c-9e38-1e03cc3d4be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:26.177087 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e70c00-0b0e-458c-9e38-1e03cc3d4be3 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:26.177365 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e70c00-0b0e-458c-9e38-1e03cc3d4be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:26.177636 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e70c00-0b0e-458c-9e38-1e03cc3d4be3 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:26.178290 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e70c00-0b0e-458c-9e38-1e03cc3d4be3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:26.180206 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e70c00-0b0e-458c-9e38-1e03cc3d4be3 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:26.201012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1224/2422] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:40:26 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:40:26.205089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33703093-9dc8-4b87-94c1-51253c36542c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:26.215810 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33703093-9dc8-4b87-94c1-51253c36542c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:26.220354 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29650bc2-fdd6-4831-a79e-054b590ec141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:26.220943 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29650bc2-fdd6-4831-a79e-054b590ec141 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:26.221221 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29650bc2-fdd6-4831-a79e-054b590ec141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:26.221490 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29650bc2-fdd6-4831-a79e-054b590ec141 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:26.222128 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29650bc2-fdd6-4831-a79e-054b590ec141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:26.223734 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29650bc2-fdd6-4831-a79e-054b590ec141 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:26.234248 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1199/2423] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:40:26.237933 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e70c00-0b0e-458c-9e38-1e03cc3d4be3 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:26.247958 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e70c00-0b0e-458c-9e38-1e03cc3d4be3 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:26.252559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d3882-fb15-45d4-a456-39f62a096f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:26.253142 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d3882-fb15-45d4-a456-39f62a096f97 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:26.253425 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d3882-fb15-45d4-a456-39f62a096f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:26.253701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d3882-fb15-45d4-a456-39f62a096f97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:26.254383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d3882-fb15-45d4-a456-39f62a096f97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:26.255959 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d3882-fb15-45d4-a456-39f62a096f97 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:26.266348 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1225/2424] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:40:26.270520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29650bc2-fdd6-4831-a79e-054b590ec141 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:26.281009 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29650bc2-fdd6-4831-a79e-054b590ec141 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:26.285653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566611e-ece7-4e7f-920e-be5983494a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:26.286336 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566611e-ece7-4e7f-920e-be5983494a25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:26.286622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566611e-ece7-4e7f-920e-be5983494a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:26.286900 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566611e-ece7-4e7f-920e-be5983494a25 admin admin] PATH_INFO: `/v3/projects/49b52a22474544d9a239d23ed90f7a54/users/72633073b3b148a99e37c87b585fd832/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:26.303620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0566611e-ece7-4e7f-920e-be5983494a25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49b52a22474544d9a239d23ed90f7a54, user_id=72633073b3b148a99e37c87b585fd832, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:26.306509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0566611e-ece7-4e7f-920e-be5983494a25 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:26.319710 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1200/2425] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:40:26 2022] PUT /identity/v3/projects/49b52a22474544d9a239d23ed90f7a54/users/72633073b3b148a99e37c87b585fd832/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:26.331038 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0622de9-681f-4871-96f4-75751fed9347 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:26.331346 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0622de9-681f-4871-96f4-75751fed9347 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:26.331622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0622de9-681f-4871-96f4-75751fed9347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:26.359648 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b0622de9-681f-4871-96f4-75751fed9347 None None] MFA Rules not processed for user `72633073b3b148a99e37c87b585fd832`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:26.360856 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b0622de9-681f-4871-96f4-75751fed9347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:26.423482 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1226/2426] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:40:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:26.434128 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566611e-ece7-4e7f-920e-be5983494a25 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:26.435847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0566611e-ece7-4e7f-920e-be5983494a25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:26.491785 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566611e-ece7-4e7f-920e-be5983494a25 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:26.495430 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dfa634-5d86-468a-958f-f51122df1b65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:26.496026 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfa634-5d86-468a-958f-f51122df1b65 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:26.496309 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfa634-5d86-468a-958f-f51122df1b65 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:26.496585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfa634-5d86-468a-958f-f51122df1b65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:26.504719 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dfa634-5d86-468a-958f-f51122df1b65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:26.506156 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dfa634-5d86-468a-958f-f51122df1b65 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:26.515433 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1201/2427] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:40:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:40:27.873919 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cc4f8-c4f5-4b43-b6e3-9571c8b8296c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:27.874459 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cc4f8-c4f5-4b43-b6e3-9571c8b8296c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:27.874583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cc4f8-c4f5-4b43-b6e3-9571c8b8296c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:27.893032 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-4c2cc4f8-c4f5-4b43-b6e3-9571c8b8296c None None] MFA Rules not processed for user `72633073b3b148a99e37c87b585fd832`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:27.894574 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4c2cc4f8-c4f5-4b43-b6e3-9571c8b8296c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:27.902401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1227/2428] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:40:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:36.511432 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:37.887838 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:44.181656 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dfa634-5d86-468a-958f-f51122df1b65 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:44.183195 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-28dfa634-5d86-468a-958f-f51122df1b65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:44.240692 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dfa634-5d86-468a-958f-f51122df1b65 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:44.244424 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e561fa-1074-4d3b-93ff-3235bec79985 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:44.245071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e561fa-1074-4d3b-93ff-3235bec79985 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:44.245338 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e561fa-1074-4d3b-93ff-3235bec79985 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:44.245615 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e561fa-1074-4d3b-93ff-3235bec79985 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:44.247381 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-17e561fa-1074-4d3b-93ff-3235bec79985 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:44.337253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e561fa-1074-4d3b-93ff-3235bec79985 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:44.338981 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e561fa-1074-4d3b-93ff-3235bec79985 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:44.361627 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1202/2429] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:40:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:40:44.959670 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2cc4f8-c4f5-4b43-b6e3-9571c8b8296c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:44.969116 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2cc4f8-c4f5-4b43-b6e3-9571c8b8296c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:44.972639 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab4046-7a82-4d21-bb71-9fad22d5e5d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:44.973240 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab4046-7a82-4d21-bb71-9fad22d5e5d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:44.973519 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab4046-7a82-4d21-bb71-9fad22d5e5d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:44.973794 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab4046-7a82-4d21-bb71-9fad22d5e5d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:44.976163 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f8ab4046-7a82-4d21-bb71-9fad22d5e5d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:44.989598 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ab4046-7a82-4d21-bb71-9fad22d5e5d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:44.991042 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ab4046-7a82-4d21-bb71-9fad22d5e5d6 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:45.001873 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1228/2430] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:40:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:40:47.409572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e05e76-6921-49e2-9e54-e72934e72fa4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:47.410066 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e05e76-6921-49e2-9e54-e72934e72fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:47.410170 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e05e76-6921-49e2-9e54-e72934e72fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:47.439081 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e5e05e76-6921-49e2-9e54-e72934e72fa4 None None] MFA Rules not processed for user `dd7b98536fa445cebd552fcbf54b757d`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:47.440256 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e5e05e76-6921-49e2-9e54-e72934e72fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:47.493539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1203/2431] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Sep 1 16:40:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:47.539782 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab4046-7a82-4d21-bb71-9fad22d5e5d6 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:47.541255 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f8ab4046-7a82-4d21-bb71-9fad22d5e5d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:47.612633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab4046-7a82-4d21-bb71-9fad22d5e5d6 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:47.618203 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd27acb7-df11-4eb6-85ba-c0241cd49138 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e789695517c1437fb224f5a6195655f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:47.618890 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27acb7-df11-4eb6-85ba-c0241cd49138 service swift] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:47.619166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27acb7-df11-4eb6-85ba-c0241cd49138 service swift] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:47.619439 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27acb7-df11-4eb6-85ba-c0241cd49138 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:47.627527 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd27acb7-df11-4eb6-85ba-c0241cd49138 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:47.629092 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd27acb7-df11-4eb6-85ba-c0241cd49138 service swift] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:47.638142 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1229/2432] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Sep 1 16:40:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:40:48.146829 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdd9eb-8c7d-4afe-a8ad-4b728216ed3e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:48.147120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdd9eb-8c7d-4afe-a8ad-4b728216ed3e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:48.147417 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdd9eb-8c7d-4afe-a8ad-4b728216ed3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:48.168223 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-53cdd9eb-8c7d-4afe-a8ad-4b728216ed3e None None] MFA Rules not processed for user `dd7b98536fa445cebd552fcbf54b757d`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:48.169423 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-53cdd9eb-8c7d-4afe-a8ad-4b728216ed3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:48.176462 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1204/2433] 10.222.0.58 () {58 vars in 967 bytes} [Thu Sep 1 16:40:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:48.188654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd27acb7-df11-4eb6-85ba-c0241cd49138 service swift] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:48.199583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd27acb7-df11-4eb6-85ba-c0241cd49138 service swift] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:48.204500 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb74571c-f56c-47fe-beff-72a4c46ea6bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e789695517c1437fb224f5a6195655f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:48.205080 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74571c-f56c-47fe-beff-72a4c46ea6bc service swift] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:48.205358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74571c-f56c-47fe-beff-72a4c46ea6bc service swift] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:48.205627 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74571c-f56c-47fe-beff-72a4c46ea6bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:48.214416 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb74571c-f56c-47fe-beff-72a4c46ea6bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:48.216007 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb74571c-f56c-47fe-beff-72a4c46ea6bc service swift] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:48.223956 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1230/2434] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Sep 1 16:40:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:40:48.433060 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7c466-8ab2-4e16-bc3d-6c90c21c90b1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:48.433523 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7c466-8ab2-4e16-bc3d-6c90c21c90b1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:48.433714 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7c466-8ab2-4e16-bc3d-6c90c21c90b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:48.442045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:48.454590 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c4f7c466-8ab2-4e16-bc3d-6c90c21c90b1 None None] MFA Rules not processed for user `dd7b98536fa445cebd552fcbf54b757d`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:48.455807 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c4f7c466-8ab2-4e16-bc3d-6c90c21c90b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:48.463321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1205/2435] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Sep 1 16:40:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:48.475607 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb74571c-f56c-47fe-beff-72a4c46ea6bc service swift] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:48.485911 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb74571c-f56c-47fe-beff-72a4c46ea6bc service swift] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:48.491394 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e6225-1581-4721-bca0-1c7531b25950 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e789695517c1437fb224f5a6195655f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:48.491981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e6225-1581-4721-bca0-1c7531b25950 service swift] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:48.492264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e6225-1581-4721-bca0-1c7531b25950 service swift] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:48.492541 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e6225-1581-4721-bca0-1c7531b25950 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:48.500368 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e6225-1581-4721-bca0-1c7531b25950 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:48.501924 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e6225-1581-4721-bca0-1c7531b25950 service swift] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:48.509783 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1231/2436] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Sep 1 16:40:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:40:48.819128 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c11b8-2271-4fc8-aaf2-410eabc53784 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:48.819442 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c11b8-2271-4fc8-aaf2-410eabc53784 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:48.819748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c11b8-2271-4fc8-aaf2-410eabc53784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:48.839871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-db4c11b8-2271-4fc8-aaf2-410eabc53784 None None] MFA Rules not processed for user `dd7b98536fa445cebd552fcbf54b757d`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:48.841178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-db4c11b8-2271-4fc8-aaf2-410eabc53784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:48.848487 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1206/2437] 10.222.0.58 () {58 vars in 967 bytes} [Thu Sep 1 16:40:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:48.861069 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e6225-1581-4721-bca0-1c7531b25950 service swift] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:48.871646 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e6225-1581-4721-bca0-1c7531b25950 service swift] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:48.876584 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb868a23-7bac-49c5-a491-7fac080eb83a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e789695517c1437fb224f5a6195655f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:48.877169 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb868a23-7bac-49c5-a491-7fac080eb83a service swift] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:48.877448 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb868a23-7bac-49c5-a491-7fac080eb83a service swift] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:48.877750 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb868a23-7bac-49c5-a491-7fac080eb83a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:48.885544 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb868a23-7bac-49c5-a491-7fac080eb83a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:48.887131 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb868a23-7bac-49c5-a491-7fac080eb83a service swift] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:48.895220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1232/2438] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Sep 1 16:40:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:40:50.997289 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c11b8-2271-4fc8-aaf2-410eabc53784 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:51.060474 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c11b8-2271-4fc8-aaf2-410eabc53784 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:51.065278 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fc315-5e75-422b-b5ef-01d89ef0933f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:51.065942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fc315-5e75-422b-b5ef-01d89ef0933f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:51.066245 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fc315-5e75-422b-b5ef-01d89ef0933f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:51.066548 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fc315-5e75-422b-b5ef-01d89ef0933f admin admin] PATH_INFO: `/v3/users/72633073b3b148a99e37c87b585fd832` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:51.067791 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7fc315-5e75-422b-b5ef-01d89ef0933f admin admin] RBAC: Authorizing `identity:delete_user(user_id=72633073b3b148a99e37c87b585fd832)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:51.069783 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7fc315-5e75-422b-b5ef-01d89ef0933f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:51.116968 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cb7fc315-5e75-422b-b5ef-01d89ef0933f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72633073b3b148a99e37c87b585fd832'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:51.123811 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cb7fc315-5e75-422b-b5ef-01d89ef0933f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72633073b3b148a99e37c87b585fd832'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:51.124468 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cb7fc315-5e75-422b-b5ef-01d89ef0933f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72633073b3b148a99e37c87b585fd832'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:51.133153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cb7fc315-5e75-422b-b5ef-01d89ef0933f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72633073b3b148a99e37c87b585fd832'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:51.145545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cb7fc315-5e75-422b-b5ef-01d89ef0933f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72633073b3b148a99e37c87b585fd832'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:51.151680 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1207/2439] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:40:50 2022] DELETE /identity/v3/users/72633073b3b148a99e37c87b585fd832 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:51.156215 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d456b-2522-400b-9966-43f11c30da84 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:51.156693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d456b-2522-400b-9966-43f11c30da84 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:51.156918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d456b-2522-400b-9966-43f11c30da84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:51.175728 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-3f6d456b-2522-400b-9966-43f11c30da84 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:51.177420 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3f6d456b-2522-400b-9966-43f11c30da84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:51.234901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1233/2440] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:40:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:40:51.244517 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fc315-5e75-422b-b5ef-01d89ef0933f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:51.245938 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cb7fc315-5e75-422b-b5ef-01d89ef0933f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:51.304749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fc315-5e75-422b-b5ef-01d89ef0933f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:51.308479 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa455a4-09e1-4a32-a008-1b4fa5bdbc2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:51.309074 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa455a4-09e1-4a32-a008-1b4fa5bdbc2f service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:51.309351 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa455a4-09e1-4a32-a008-1b4fa5bdbc2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:51.309620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa455a4-09e1-4a32-a008-1b4fa5bdbc2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:51.319504 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa455a4-09e1-4a32-a008-1b4fa5bdbc2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:51.320938 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa455a4-09e1-4a32-a008-1b4fa5bdbc2f service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:51.330398 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1208/2441] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:40:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:40:51.559613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d456b-2522-400b-9966-43f11c30da84 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:51.561464 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3f6d456b-2522-400b-9966-43f11c30da84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:51.574422 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d456b-2522-400b-9966-43f11c30da84 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:51.579336 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a071b-6636-48ea-8c82-14d079225f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:51.579935 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a071b-6636-48ea-8c82-14d079225f3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:51.580218 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a071b-6636-48ea-8c82-14d079225f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:51.580490 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a071b-6636-48ea-8c82-14d079225f3f admin admin] PATH_INFO: `/v3/projects/49b52a22474544d9a239d23ed90f7a54` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:51.581609 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a071b-6636-48ea-8c82-14d079225f3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=49b52a22474544d9a239d23ed90f7a54)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:51.583608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a071b-6636-48ea-8c82-14d079225f3f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:51.622685 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:51.636975 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d55a071b-6636-48ea-8c82-14d079225f3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49b52a22474544d9a239d23ed90f7a54', 'request_id': 'req-d55a071b-6636-48ea-8c82-14d079225f3f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:51.643708 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d55a071b-6636-48ea-8c82-14d079225f3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49b52a22474544d9a239d23ed90f7a54', 'request_id': 'req-d55a071b-6636-48ea-8c82-14d079225f3f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:51.649435 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d55a071b-6636-48ea-8c82-14d079225f3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49b52a22474544d9a239d23ed90f7a54', 'request_id': 'req-d55a071b-6636-48ea-8c82-14d079225f3f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:51.661485 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d55a071b-6636-48ea-8c82-14d079225f3f admin admin] The token cache is being invalidate because project 49b52a22474544d9a239d23ed90f7a54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:40:51.661774 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d55a071b-6636-48ea-8c82-14d079225f3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:51.662417 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d55a071b-6636-48ea-8c82-14d079225f3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:51.663893 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1234/2442] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:40:51 2022] DELETE /identity/v3/projects/49b52a22474544d9a239d23ed90f7a54 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:51.668853 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09358954-f30e-4452-a0f4-da11eb5e2173 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:51.669241 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09358954-f30e-4452-a0f4-da11eb5e2173 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:51.669589 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09358954-f30e-4452-a0f4-da11eb5e2173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:51.689479 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-09358954-f30e-4452-a0f4-da11eb5e2173 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:51.690719 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-09358954-f30e-4452-a0f4-da11eb5e2173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:51.699728 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:51.752664 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1209/2443] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:40:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:51.760986 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720fcce5-c831-4559-b7f0-6ea68405cf2a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:51.761270 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720fcce5-c831-4559-b7f0-6ea68405cf2a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:51.761546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720fcce5-c831-4559-b7f0-6ea68405cf2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:51.781200 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-720fcce5-c831-4559-b7f0-6ea68405cf2a None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:51.782370 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-720fcce5-c831-4559-b7f0-6ea68405cf2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:51.790116 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1235/2444] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:40:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:51.794136 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09358954-f30e-4452-a0f4-da11eb5e2173 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:51.804190 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09358954-f30e-4452-a0f4-da11eb5e2173 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:51.808680 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8755ae-0a83-4583-bb79-93d195e6dff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:51.809271 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8755ae-0a83-4583-bb79-93d195e6dff8 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:51.809554 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8755ae-0a83-4583-bb79-93d195e6dff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:51.809828 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8755ae-0a83-4583-bb79-93d195e6dff8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:51.810508 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8755ae-0a83-4583-bb79-93d195e6dff8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:40:51.810814 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8755ae-0a83-4583-bb79-93d195e6dff8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:51.812383 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8755ae-0a83-4583-bb79-93d195e6dff8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:51.824264 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1210/2445] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:40:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:40:51.828576 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720fcce5-c831-4559-b7f0-6ea68405cf2a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:51.837775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720fcce5-c831-4559-b7f0-6ea68405cf2a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:51.842570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344f703-f55b-49bc-9219-c62070644b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:51.843157 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344f703-f55b-49bc-9219-c62070644b8c admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:51.843439 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344f703-f55b-49bc-9219-c62070644b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:51.843714 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344f703-f55b-49bc-9219-c62070644b8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:51.844338 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344f703-f55b-49bc-9219-c62070644b8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:51.846238 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344f703-f55b-49bc-9219-c62070644b8c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:51.883928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1236/2446] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:40:51 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:40:51.888026 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8755ae-0a83-4583-bb79-93d195e6dff8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:51.898352 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8755ae-0a83-4583-bb79-93d195e6dff8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:51.959442 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e616ea-1cd5-4eda-b568-fa71a349281b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:51.960335 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e616ea-1cd5-4eda-b568-fa71a349281b admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:51.960761 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e616ea-1cd5-4eda-b568-fa71a349281b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:51.961178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e616ea-1cd5-4eda-b568-fa71a349281b admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:51.962144 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e616ea-1cd5-4eda-b568-fa71a349281b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:51.965241 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e616ea-1cd5-4eda-b568-fa71a349281b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:51.990912 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1211/2447] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:40:51 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:40:51.995057 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344f703-f55b-49bc-9219-c62070644b8c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:52.005372 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344f703-f55b-49bc-9219-c62070644b8c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:52.009848 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754f913d-57f1-43cb-956a-49dd2bedd153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:52.010466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f913d-57f1-43cb-956a-49dd2bedd153 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:52.010742 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f913d-57f1-43cb-956a-49dd2bedd153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:52.011009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f913d-57f1-43cb-956a-49dd2bedd153 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:52.011646 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754f913d-57f1-43cb-956a-49dd2bedd153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:52.013248 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754f913d-57f1-43cb-956a-49dd2bedd153 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:52.023223 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1237/2448] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:40:52.027099 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e616ea-1cd5-4eda-b568-fa71a349281b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:52.037639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e616ea-1cd5-4eda-b568-fa71a349281b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:52.042365 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8de1a0-d2a5-439a-bc3d-2f33c773cfcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:52.042968 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8de1a0-d2a5-439a-bc3d-2f33c773cfcc admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:52.043253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8de1a0-d2a5-439a-bc3d-2f33c773cfcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:52.043531 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8de1a0-d2a5-439a-bc3d-2f33c773cfcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:52.044178 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8de1a0-d2a5-439a-bc3d-2f33c773cfcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:52.045776 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8de1a0-d2a5-439a-bc3d-2f33c773cfcc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:52.058227 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1212/2449] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:40:52.062546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754f913d-57f1-43cb-956a-49dd2bedd153 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:52.072516 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754f913d-57f1-43cb-956a-49dd2bedd153 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:52.077430 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768db530-b429-4b78-b90d-6bbfd687b686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:52.078070 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768db530-b429-4b78-b90d-6bbfd687b686 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:52.078383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768db530-b429-4b78-b90d-6bbfd687b686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:52.078664 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768db530-b429-4b78-b90d-6bbfd687b686 admin admin] PATH_INFO: `/v3/projects/bde90cf98c1140f298e9cc8db9c8932f/users/d0b60450d50b4fbaa9a85dd39643e403/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:52.093347 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768db530-b429-4b78-b90d-6bbfd687b686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bde90cf98c1140f298e9cc8db9c8932f, user_id=d0b60450d50b4fbaa9a85dd39643e403, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:52.096025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768db530-b429-4b78-b90d-6bbfd687b686 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:52.109448 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1238/2450] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:40:52 2022] PUT /identity/v3/projects/bde90cf98c1140f298e9cc8db9c8932f/users/d0b60450d50b4fbaa9a85dd39643e403/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:52.115204 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d096ed-0b8e-486b-bd08-0c6d8f3f2ac9 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:52.115492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d096ed-0b8e-486b-bd08-0c6d8f3f2ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:52.115776 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d096ed-0b8e-486b-bd08-0c6d8f3f2ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:52.134420 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-32d096ed-0b8e-486b-bd08-0c6d8f3f2ac9 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:52.135625 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-32d096ed-0b8e-486b-bd08-0c6d8f3f2ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:52.197150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1213/2451] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:40:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:52.206747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768db530-b429-4b78-b90d-6bbfd687b686 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:52.208665 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-768db530-b429-4b78-b90d-6bbfd687b686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:52.273205 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768db530-b429-4b78-b90d-6bbfd687b686 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:52.276836 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cab674-3701-4adb-ac36-8fefc7c8566a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:52.277441 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cab674-3701-4adb-ac36-8fefc7c8566a service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:52.277718 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cab674-3701-4adb-ac36-8fefc7c8566a service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:52.277985 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cab674-3701-4adb-ac36-8fefc7c8566a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:52.288930 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cab674-3701-4adb-ac36-8fefc7c8566a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:52.290339 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cab674-3701-4adb-ac36-8fefc7c8566a service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:52.299943 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1239/2452] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:40:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:40:56.389111 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c9f0f2-2efd-44d9-b42a-ba41f93ec21c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:56.389916 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c9f0f2-2efd-44d9-b42a-ba41f93ec21c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:56.389916 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c9f0f2-2efd-44d9-b42a-ba41f93ec21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:56.441643 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-47c9f0f2-2efd-44d9-b42a-ba41f93ec21c None None] MFA Rules not processed for user `d0b60450d50b4fbaa9a85dd39643e403`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:56.443595 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-47c9f0f2-2efd-44d9-b42a-ba41f93ec21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:56.454222 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:56.555684 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1214/2453] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:40:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:56.560277 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cab674-3701-4adb-ac36-8fefc7c8566a service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:56.570033 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cab674-3701-4adb-ac36-8fefc7c8566a service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:56.574816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5cfd8-79d0-42b0-8660-af4945de584e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:56.575487 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5cfd8-79d0-42b0-8660-af4945de584e admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:56.575831 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5cfd8-79d0-42b0-8660-af4945de584e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:56.576191 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5cfd8-79d0-42b0-8660-af4945de584e admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:56.576900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf5cfd8-79d0-42b0-8660-af4945de584e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:56.585972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf5cfd8-79d0-42b0-8660-af4945de584e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:56.621388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1240/2454] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:40:56 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:40:56.625672 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c9f0f2-2efd-44d9-b42a-ba41f93ec21c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:56.637242 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c9f0f2-2efd-44d9-b42a-ba41f93ec21c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:56.698317 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345dd59-7352-4e2a-a00e-ee41882a8029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:56.699033 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345dd59-7352-4e2a-a00e-ee41882a8029 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:56.699388 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345dd59-7352-4e2a-a00e-ee41882a8029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:56.699740 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345dd59-7352-4e2a-a00e-ee41882a8029 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:56.700478 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8345dd59-7352-4e2a-a00e-ee41882a8029 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:56.702461 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8345dd59-7352-4e2a-a00e-ee41882a8029 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:56.725373 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1215/2455] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:40:56 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:40:56.730979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5cfd8-79d0-42b0-8660-af4945de584e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:56.747704 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5cfd8-79d0-42b0-8660-af4945de584e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:56.755339 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbed016-b177-414c-88cf-a5abdcdb8be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:56.756532 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbed016-b177-414c-88cf-a5abdcdb8be0 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:56.757119 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbed016-b177-414c-88cf-a5abdcdb8be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:56.757686 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbed016-b177-414c-88cf-a5abdcdb8be0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:56.758995 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbed016-b177-414c-88cf-a5abdcdb8be0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:56.761787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbed016-b177-414c-88cf-a5abdcdb8be0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:56.775686 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1241/2456] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:40:56.779953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345dd59-7352-4e2a-a00e-ee41882a8029 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:56.790944 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345dd59-7352-4e2a-a00e-ee41882a8029 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:56.795701 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68660f65-2619-41d5-a98f-3701ca201e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:56.796369 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68660f65-2619-41d5-a98f-3701ca201e36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:56.796654 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68660f65-2619-41d5-a98f-3701ca201e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:56.796934 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68660f65-2619-41d5-a98f-3701ca201e36 admin admin] PATH_INFO: `/v3/projects/61201dd344504df4958112b9b6eff04c/users/5b4ce389b7714919abd2111411c36651/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:56.813767 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68660f65-2619-41d5-a98f-3701ca201e36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61201dd344504df4958112b9b6eff04c, user_id=5b4ce389b7714919abd2111411c36651, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:56.816489 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68660f65-2619-41d5-a98f-3701ca201e36 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:56.830941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1216/2457] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:40:56 2022] PUT /identity/v3/projects/61201dd344504df4958112b9b6eff04c/users/5b4ce389b7714919abd2111411c36651/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:56.835369 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbed016-b177-414c-88cf-a5abdcdb8be0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:56.845618 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbed016-b177-414c-88cf-a5abdcdb8be0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:56.859127 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:56.907913 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02191e1-78e5-45bd-ba28-4f1f5b95d61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:56.908530 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02191e1-78e5-45bd-ba28-4f1f5b95d61a admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:56.908823 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02191e1-78e5-45bd-ba28-4f1f5b95d61a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:56.909087 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02191e1-78e5-45bd-ba28-4f1f5b95d61a admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:56.909751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02191e1-78e5-45bd-ba28-4f1f5b95d61a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:56.912054 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02191e1-78e5-45bd-ba28-4f1f5b95d61a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:56.922822 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1242/2458] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:40:56.927821 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68660f65-2619-41d5-a98f-3701ca201e36 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:56.939537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68660f65-2619-41d5-a98f-3701ca201e36 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:56.944881 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee54197-76de-49d5-a669-c950bb43dea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:56.945602 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee54197-76de-49d5-a669-c950bb43dea8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:56.945889 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee54197-76de-49d5-a669-c950bb43dea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:56.946171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee54197-76de-49d5-a669-c950bb43dea8 admin admin] PATH_INFO: `/v3/projects/61201dd344504df4958112b9b6eff04c/users/5b4ce389b7714919abd2111411c36651/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:56.948462 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee54197-76de-49d5-a669-c950bb43dea8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61201dd344504df4958112b9b6eff04c, user_id=5b4ce389b7714919abd2111411c36651, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:56.951332 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee54197-76de-49d5-a669-c950bb43dea8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:56.965776 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1217/2459] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:40:56 2022] PUT /identity/v3/projects/61201dd344504df4958112b9b6eff04c/users/5b4ce389b7714919abd2111411c36651/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:01.406471 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c630c5-8aa4-4921-b280-ba03bf545b05 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:01.406965 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c630c5-8aa4-4921-b280-ba03bf545b05 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:01.407176 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c630c5-8aa4-4921-b280-ba03bf545b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:01.437884 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-86c630c5-8aa4-4921-b280-ba03bf545b05 None None] MFA Rules not processed for user `5b4ce389b7714919abd2111411c36651`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:01.440305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-86c630c5-8aa4-4921-b280-ba03bf545b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:01.530745 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1243/2460] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:41:01 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:01.539991 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee54197-76de-49d5-a669-c950bb43dea8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:01.542179 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dee54197-76de-49d5-a669-c950bb43dea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:01.602449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:01.620056 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee54197-76de-49d5-a669-c950bb43dea8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:01.625683 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ff24d-1c16-46fc-b1cb-e6db804d32a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:01.626306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ff24d-1c16-46fc-b1cb-e6db804d32a6 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:01.626589 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ff24d-1c16-46fc-b1cb-e6db804d32a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:01.626861 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ff24d-1c16-46fc-b1cb-e6db804d32a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:01.688048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ff24d-1c16-46fc-b1cb-e6db804d32a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:01.689562 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ff24d-1c16-46fc-b1cb-e6db804d32a6 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:01.699226 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1218/2461] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:02.861936 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e624bbc-cfbd-4725-af07-7ade209eebb4 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:02.862463 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e624bbc-cfbd-4725-af07-7ade209eebb4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:02.862805 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e624bbc-cfbd-4725-af07-7ade209eebb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:02.887703 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-3e624bbc-cfbd-4725-af07-7ade209eebb4 None None] MFA Rules not processed for user `d0b60450d50b4fbaa9a85dd39643e403`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:02.888945 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3e624bbc-cfbd-4725-af07-7ade209eebb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:02.897052 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1244/2462] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:41:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:03.290224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632e460-cace-4f1e-a330-4ab44c72b507 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:03.290868 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632e460-cace-4f1e-a330-4ab44c72b507 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:03.290868 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632e460-cace-4f1e-a330-4ab44c72b507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:03.312434 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2632e460-cace-4f1e-a330-4ab44c72b507 None None] MFA Rules not processed for user `dd7b98536fa445cebd552fcbf54b757d`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:03.314331 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2632e460-cace-4f1e-a330-4ab44c72b507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:03.383470 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1219/2463] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Sep 1 16:41:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:03.396584 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e624bbc-cfbd-4725-af07-7ade209eebb4 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:03.398284 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3e624bbc-cfbd-4725-af07-7ade209eebb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:03.480557 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e624bbc-cfbd-4725-af07-7ade209eebb4 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:03.486720 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e51fe6-bf01-47af-8d73-31883969ab18 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e789695517c1437fb224f5a6195655f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:03.487459 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e51fe6-bf01-47af-8d73-31883969ab18 service swift] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:03.487938 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e51fe6-bf01-47af-8d73-31883969ab18 service swift] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:03.488237 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e51fe6-bf01-47af-8d73-31883969ab18 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:03.498954 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e51fe6-bf01-47af-8d73-31883969ab18 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:03.502323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e51fe6-bf01-47af-8d73-31883969ab18 service swift] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:03.515005 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1245/2464] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Sep 1 16:41:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:41:13.375059 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=53 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:13.511683 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:17.455144 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632e460-cace-4f1e-a330-4ab44c72b507 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:17.456665 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2632e460-cace-4f1e-a330-4ab44c72b507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:17.512725 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632e460-cace-4f1e-a330-4ab44c72b507 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:17.516310 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a83e0-7538-4408-af86-08e3f577663b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:17.517192 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a83e0-7538-4408-af86-08e3f577663b service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:17.517472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a83e0-7538-4408-af86-08e3f577663b service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:17.517744 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a83e0-7538-4408-af86-08e3f577663b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:17.520019 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-869a83e0-7538-4408-af86-08e3f577663b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:17.529458 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a83e0-7538-4408-af86-08e3f577663b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:17.530910 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a83e0-7538-4408-af86-08e3f577663b service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:17.539181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1220/2465] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:21.717592 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e51fe6-bf01-47af-8d73-31883969ab18 service swift] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:21.719148 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-03e51fe6-bf01-47af-8d73-31883969ab18 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:21.786140 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e51fe6-bf01-47af-8d73-31883969ab18 service swift] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:21.793406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45fab8-bcd7-4085-8de5-e25d9d9df507 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2c872129414105a112661cc2b477e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:21.794773 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45fab8-bcd7-4085-8de5-e25d9d9df507 service glance] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:21.795259 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45fab8-bcd7-4085-8de5-e25d9d9df507 service glance] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:21.795738 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45fab8-bcd7-4085-8de5-e25d9d9df507 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:21.798589 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8e45fab8-bcd7-4085-8de5-e25d9d9df507 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:21.814411 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45fab8-bcd7-4085-8de5-e25d9d9df507 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:21.816542 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45fab8-bcd7-4085-8de5-e25d9d9df507 service glance] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:21.827812 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1246/2466] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:21.896202 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a83e0-7538-4408-af86-08e3f577663b service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:21.905131 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a83e0-7538-4408-af86-08e3f577663b service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:21.909097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a67bd1-ee72-49b4-a8bb-f9ecd0a04228 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:21.909729 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a67bd1-ee72-49b4-a8bb-f9ecd0a04228 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:21.910013 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a67bd1-ee72-49b4-a8bb-f9ecd0a04228 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:21.910325 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a67bd1-ee72-49b4-a8bb-f9ecd0a04228 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:21.912050 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-41a67bd1-ee72-49b4-a8bb-f9ecd0a04228 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:21.921576 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a67bd1-ee72-49b4-a8bb-f9ecd0a04228 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:21.923053 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a67bd1-ee72-49b4-a8bb-f9ecd0a04228 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:21.931434 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1221/2467] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:22.006127 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45fab8-bcd7-4085-8de5-e25d9d9df507 service glance] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:22.015466 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45fab8-bcd7-4085-8de5-e25d9d9df507 service glance] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:22.019520 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf471a6-e960-48bd-9400-73a99805f02f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:22.020117 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf471a6-e960-48bd-9400-73a99805f02f service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:22.020400 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf471a6-e960-48bd-9400-73a99805f02f service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:22.020674 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf471a6-e960-48bd-9400-73a99805f02f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:22.022510 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9bf471a6-e960-48bd-9400-73a99805f02f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:22.036344 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf471a6-e960-48bd-9400-73a99805f02f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:22.037749 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf471a6-e960-48bd-9400-73a99805f02f service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:22.048691 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1247/2468] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:23.936391 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2589b-308e-4c42-95ef-93294b875372 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:23.936869 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2589b-308e-4c42-95ef-93294b875372 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:23.937061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2589b-308e-4c42-95ef-93294b875372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:23.985925 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-88d2589b-308e-4c42-95ef-93294b875372 None None] MFA Rules not processed for user `dd7b98536fa445cebd552fcbf54b757d`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:23.987182 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-88d2589b-308e-4c42-95ef-93294b875372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:24.003391 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1222/2469] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Sep 1 16:41:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:24.015146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf471a6-e960-48bd-9400-73a99805f02f service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:24.037189 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf471a6-e960-48bd-9400-73a99805f02f service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:24.042245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa2edf-d890-4169-a2d0-6c47a273d19d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e789695517c1437fb224f5a6195655f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:24.042850 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa2edf-d890-4169-a2d0-6c47a273d19d service swift] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:24.043175 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa2edf-d890-4169-a2d0-6c47a273d19d service swift] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:24.043452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa2edf-d890-4169-a2d0-6c47a273d19d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:24.052344 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa2edf-d890-4169-a2d0-6c47a273d19d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:24.054166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa2edf-d890-4169-a2d0-6c47a273d19d service swift] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:24.062667 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1248/2470] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Sep 1 16:41:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:41:34.000505 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=50 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:34.059151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=42 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:50.102089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f100e1-2bc9-4ff5-acef-50abbd308116 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:50.104369 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f100e1-2bc9-4ff5-acef-50abbd308116 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:50.104678 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f100e1-2bc9-4ff5-acef-50abbd308116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:50.125224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-25f100e1-2bc9-4ff5-acef-50abbd308116 None None] MFA Rules not processed for user `dd7b98536fa445cebd552fcbf54b757d`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:50.126451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-25f100e1-2bc9-4ff5-acef-50abbd308116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:50.133485 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1223/2471] 10.222.0.58 () {58 vars in 967 bytes} [Thu Sep 1 16:41:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:50.156851 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa2edf-d890-4169-a2d0-6c47a273d19d service swift] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:50.168295 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa2edf-d890-4169-a2d0-6c47a273d19d service swift] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:50.173365 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de2fb3-5a71-4519-9b69-441560ea17ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e789695517c1437fb224f5a6195655f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:50.174080 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de2fb3-5a71-4519-9b69-441560ea17ee service swift] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:50.174446 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de2fb3-5a71-4519-9b69-441560ea17ee service swift] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:50.174861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de2fb3-5a71-4519-9b69-441560ea17ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:50.183100 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8de2fb3-5a71-4519-9b69-441560ea17ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:50.184741 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8de2fb3-5a71-4519-9b69-441560ea17ee service swift] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:50.192905 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1249/2472] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Sep 1 16:41:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:42:00.119741 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:00.189516 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=6 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:01.278917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f100e1-2bc9-4ff5-acef-50abbd308116 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:01.288187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f100e1-2bc9-4ff5-acef-50abbd308116 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:01.291951 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5760247-8d0e-445f-91f2-17e4a5ce68d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:01.292545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5760247-8d0e-445f-91f2-17e4a5ce68d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:01.292819 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5760247-8d0e-445f-91f2-17e4a5ce68d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:01.293266 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5760247-8d0e-445f-91f2-17e4a5ce68d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:01.302271 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5760247-8d0e-445f-91f2-17e4a5ce68d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:01.306390 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5760247-8d0e-445f-91f2-17e4a5ce68d4 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:01.313995 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1224/2473] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:11.309688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:11.316183 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de2fb3-5a71-4519-9b69-441560ea17ee service swift] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:11.317582 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f8de2fb3-5a71-4519-9b69-441560ea17ee service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:11.375349 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de2fb3-5a71-4519-9b69-441560ea17ee service swift] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:11.379019 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0fac9-ad98-46aa-b37a-2ee249d8a4e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:11.379628 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0fac9-ad98-46aa-b37a-2ee249d8a4e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:11.379910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0fac9-ad98-46aa-b37a-2ee249d8a4e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:11.380187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0fac9-ad98-46aa-b37a-2ee249d8a4e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:11.381924 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5fe0fac9-ad98-46aa-b37a-2ee249d8a4e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:11.462368 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe0fac9-ad98-46aa-b37a-2ee249d8a4e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:11.463820 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe0fac9-ad98-46aa-b37a-2ee249d8a4e5 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:11.482302 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1250/2474] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:21.480095 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=61 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:25.477013 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5760247-8d0e-445f-91f2-17e4a5ce68d4 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:25.485709 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5760247-8d0e-445f-91f2-17e4a5ce68d4 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:25.489336 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0bc26-0bbf-46e4-9896-f45e3a6749d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:25.489926 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0bc26-0bbf-46e4-9896-f45e3a6749d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:25.490242 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0bc26-0bbf-46e4-9896-f45e3a6749d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:25.490520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0bc26-0bbf-46e4-9896-f45e3a6749d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:25.492208 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c9e0bc26-0bbf-46e4-9896-f45e3a6749d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:25.554782 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e0bc26-0bbf-46e4-9896-f45e3a6749d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:25.556219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e0bc26-0bbf-46e4-9896-f45e3a6749d7 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:25.566383 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1225/2475] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:28.213331 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0fac9-ad98-46aa-b37a-2ee249d8a4e5 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:28.214793 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5fe0fac9-ad98-46aa-b37a-2ee249d8a4e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:28.230553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0fac9-ad98-46aa-b37a-2ee249d8a4e5 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:28.236203 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab0cc9-bad9-41f1-bd91-65650541642c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:28.237096 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab0cc9-bad9-41f1-bd91-65650541642c service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:28.237577 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab0cc9-bad9-41f1-bd91-65650541642c service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:28.237710 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab0cc9-bad9-41f1-bd91-65650541642c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:28.239852 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-93ab0cc9-bad9-41f1-bd91-65650541642c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:28.341045 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab0cc9-bad9-41f1-bd91-65650541642c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:28.344254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab0cc9-bad9-41f1-bd91-65650541642c service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:28.363230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1251/2476] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:28.871230 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0bc26-0bbf-46e4-9896-f45e3a6749d7 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:28.872773 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c9e0bc26-0bbf-46e4-9896-f45e3a6749d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:28.969521 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0bc26-0bbf-46e4-9896-f45e3a6749d7 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:28.973177 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab0cc9-bad9-41f1-bd91-65650541642c service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:28.974609 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-93ab0cc9-bad9-41f1-bd91-65650541642c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:28.978409 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f677a06-88d3-4e13-98eb-ae86f2c1ff0a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12302cdb2854b0684179647a6dadf6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:28.979492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f677a06-88d3-4e13-98eb-ae86f2c1ff0a service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:28.980379 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f677a06-88d3-4e13-98eb-ae86f2c1ff0a service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:28.980814 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f677a06-88d3-4e13-98eb-ae86f2c1ff0a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:28.983191 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2f677a06-88d3-4e13-98eb-ae86f2c1ff0a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:28.988339 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab0cc9-bad9-41f1-bd91-65650541642c service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:28.992932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5f9e6-79bb-40b3-9247-c1bb97245435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:28.993565 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5f9e6-79bb-40b3-9247-c1bb97245435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:28.993845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5f9e6-79bb-40b3-9247-c1bb97245435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:28.994117 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5f9e6-79bb-40b3-9247-c1bb97245435 admin admin] PATH_INFO: `/v3/users/4ecfc51400414558b8389a9c35b2fa1c` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:28.995336 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f677a06-88d3-4e13-98eb-ae86f2c1ff0a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:28.995478 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb5f9e6-79bb-40b3-9247-c1bb97245435 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ecfc51400414558b8389a9c35b2fa1c)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:28.997451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f677a06-88d3-4e13-98eb-ae86f2c1ff0a service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:28.997799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb5f9e6-79bb-40b3-9247-c1bb97245435 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:29.007402 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1226/2477] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:29.057802 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1eb5f9e6-79bb-40b3-9247-c1bb97245435 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ecfc51400414558b8389a9c35b2fa1c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:29.065535 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1eb5f9e6-79bb-40b3-9247-c1bb97245435 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ecfc51400414558b8389a9c35b2fa1c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:29.066353 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1eb5f9e6-79bb-40b3-9247-c1bb97245435 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ecfc51400414558b8389a9c35b2fa1c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:29.067056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1eb5f9e6-79bb-40b3-9247-c1bb97245435 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ecfc51400414558b8389a9c35b2fa1c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:29.079801 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1eb5f9e6-79bb-40b3-9247-c1bb97245435 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ecfc51400414558b8389a9c35b2fa1c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:29.091748 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1252/2478] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:42:28 2022] DELETE /identity/v3/users/4ecfc51400414558b8389a9c35b2fa1c => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:42:29.268861 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f677a06-88d3-4e13-98eb-ae86f2c1ff0a service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:29.282539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f677a06-88d3-4e13-98eb-ae86f2c1ff0a service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:29.340488 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b9dc9-d734-4839-8aae-1077030e4148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:29.341130 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b9dc9-d734-4839-8aae-1077030e4148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:29.341411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b9dc9-d734-4839-8aae-1077030e4148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:29.341685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b9dc9-d734-4839-8aae-1077030e4148 admin admin] PATH_INFO: `/v3/projects/0b61e46f94b44e6d963e27ccae5dd165` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:29.342823 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b9dc9-d734-4839-8aae-1077030e4148 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b61e46f94b44e6d963e27ccae5dd165)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:29.344728 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b9dc9-d734-4839-8aae-1077030e4148 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:29.376440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:29.393316 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f82b9dc9-d734-4839-8aae-1077030e4148 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b61e46f94b44e6d963e27ccae5dd165', 'request_id': 'req-f82b9dc9-d734-4839-8aae-1077030e4148'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:29.399767 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f82b9dc9-d734-4839-8aae-1077030e4148 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b61e46f94b44e6d963e27ccae5dd165', 'request_id': 'req-f82b9dc9-d734-4839-8aae-1077030e4148'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:29.406784 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f82b9dc9-d734-4839-8aae-1077030e4148 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b61e46f94b44e6d963e27ccae5dd165', 'request_id': 'req-f82b9dc9-d734-4839-8aae-1077030e4148'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:29.417038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f82b9dc9-d734-4839-8aae-1077030e4148 admin admin] The token cache is being invalidate because project 0b61e46f94b44e6d963e27ccae5dd165 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:42:29.417326 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f82b9dc9-d734-4839-8aae-1077030e4148 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:29.418480 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f82b9dc9-d734-4839-8aae-1077030e4148 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:29.419956 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1227/2479] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:42:29 2022] DELETE /identity/v3/projects/0b61e46f94b44e6d963e27ccae5dd165 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:29.424811 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a839535-8feb-4b45-9276-d6636385bca0 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:29.425277 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a839535-8feb-4b45-9276-d6636385bca0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:29.425394 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a839535-8feb-4b45-9276-d6636385bca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:29.445150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-9a839535-8feb-4b45-9276-d6636385bca0 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:29.446389 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9a839535-8feb-4b45-9276-d6636385bca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:29.483918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:29.506181 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1253/2480] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:29.515472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574687a9-5a69-4fc7-8fd3-ee383e327138 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:29.515701 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574687a9-5a69-4fc7-8fd3-ee383e327138 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:29.515981 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574687a9-5a69-4fc7-8fd3-ee383e327138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:29.535790 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-574687a9-5a69-4fc7-8fd3-ee383e327138 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:29.536993 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-574687a9-5a69-4fc7-8fd3-ee383e327138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:29.545480 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1228/2481] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:29.549794 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a839535-8feb-4b45-9276-d6636385bca0 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:29.560160 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a839535-8feb-4b45-9276-d6636385bca0 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:29.564812 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ab180-b758-471a-b071-9b260cf6c86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:29.565415 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ab180-b758-471a-b071-9b260cf6c86e admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:29.565695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ab180-b758-471a-b071-9b260cf6c86e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:29.565972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ab180-b758-471a-b071-9b260cf6c86e admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:29.566666 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ab180-b758-471a-b071-9b260cf6c86e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:42:29.566975 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ab180-b758-471a-b071-9b260cf6c86e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:29.568556 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ab180-b758-471a-b071-9b260cf6c86e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:29.580281 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1254/2482] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:42:29.584430 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574687a9-5a69-4fc7-8fd3-ee383e327138 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:29.593622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574687a9-5a69-4fc7-8fd3-ee383e327138 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:29.598156 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f664a-2964-4ed2-8ef7-6819322beca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:29.598776 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f664a-2964-4ed2-8ef7-6819322beca9 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:29.599085 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f664a-2964-4ed2-8ef7-6819322beca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:29.599365 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f664a-2964-4ed2-8ef7-6819322beca9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:29.599993 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316f664a-2964-4ed2-8ef7-6819322beca9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:29.601873 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316f664a-2964-4ed2-8ef7-6819322beca9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:29.630790 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1229/2483] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:42:29 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:29.635181 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ab180-b758-471a-b071-9b260cf6c86e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:29.645748 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ab180-b758-471a-b071-9b260cf6c86e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:29.702188 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c19bd3-5efb-48ea-9e32-2dafc1378f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:29.702817 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c19bd3-5efb-48ea-9e32-2dafc1378f87 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:29.703101 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c19bd3-5efb-48ea-9e32-2dafc1378f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:29.703376 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c19bd3-5efb-48ea-9e32-2dafc1378f87 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:29.704001 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c19bd3-5efb-48ea-9e32-2dafc1378f87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:29.705914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c19bd3-5efb-48ea-9e32-2dafc1378f87 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:29.727535 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1255/2484] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:42:29 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:42:29.732223 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f664a-2964-4ed2-8ef7-6819322beca9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:29.742589 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f664a-2964-4ed2-8ef7-6819322beca9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:29.747753 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a24475-6c13-4d35-8a4b-63178399be5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:29.748347 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a24475-6c13-4d35-8a4b-63178399be5a admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:29.748630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a24475-6c13-4d35-8a4b-63178399be5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:29.748907 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a24475-6c13-4d35-8a4b-63178399be5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:29.749579 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a24475-6c13-4d35-8a4b-63178399be5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:29.751201 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a24475-6c13-4d35-8a4b-63178399be5a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:29.761081 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1230/2485] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:42:29.764942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c19bd3-5efb-48ea-9e32-2dafc1378f87 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:29.775334 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c19bd3-5efb-48ea-9e32-2dafc1378f87 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:29.780878 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6058609-8a02-4d2b-b499-6ab30d569b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:29.781622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6058609-8a02-4d2b-b499-6ab30d569b47 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:29.782056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6058609-8a02-4d2b-b499-6ab30d569b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:29.782372 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6058609-8a02-4d2b-b499-6ab30d569b47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:29.783022 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6058609-8a02-4d2b-b499-6ab30d569b47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:29.784617 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6058609-8a02-4d2b-b499-6ab30d569b47 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:29.796109 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1256/2486] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:29.800270 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a24475-6c13-4d35-8a4b-63178399be5a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:29.811045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a24475-6c13-4d35-8a4b-63178399be5a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:29.816882 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4574f2-e440-4a2c-91dc-c3058dc5344b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:29.818062 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4574f2-e440-4a2c-91dc-c3058dc5344b admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:29.818472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4574f2-e440-4a2c-91dc-c3058dc5344b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:29.818733 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4574f2-e440-4a2c-91dc-c3058dc5344b admin admin] PATH_INFO: `/v3/projects/c38be99d56404657938612fa8c06999f/users/6c5ce5557d60473f9b85373892ad5c8f/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:29.834187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4574f2-e440-4a2c-91dc-c3058dc5344b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c38be99d56404657938612fa8c06999f, user_id=6c5ce5557d60473f9b85373892ad5c8f, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:29.836855 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4574f2-e440-4a2c-91dc-c3058dc5344b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:29.851489 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1231/2487] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:29 2022] PUT /identity/v3/projects/c38be99d56404657938612fa8c06999f/users/6c5ce5557d60473f9b85373892ad5c8f/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:29.862126 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd0e3f-a6d9-4b47-ab91-4f1dd6a4f4b7 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:29.862504 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd0e3f-a6d9-4b47-ab91-4f1dd6a4f4b7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:29.862853 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd0e3f-a6d9-4b47-ab91-4f1dd6a4f4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:29.891052 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-0edd0e3f-a6d9-4b47-ab91-4f1dd6a4f4b7 None None] MFA Rules not processed for user `6c5ce5557d60473f9b85373892ad5c8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:29.892247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0edd0e3f-a6d9-4b47-ab91-4f1dd6a4f4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:29.956737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1257/2488] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:42:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:29.968724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4574f2-e440-4a2c-91dc-c3058dc5344b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:29.970550 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4f4574f2-e440-4a2c-91dc-c3058dc5344b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:30.042641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4574f2-e440-4a2c-91dc-c3058dc5344b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:30.048312 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401522a7-9634-428f-b6fb-1f34c974bac7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:30.048927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401522a7-9634-428f-b6fb-1f34c974bac7 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:30.049211 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401522a7-9634-428f-b6fb-1f34c974bac7 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:30.049489 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401522a7-9634-428f-b6fb-1f34c974bac7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:30.057631 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401522a7-9634-428f-b6fb-1f34c974bac7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:30.059752 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401522a7-9634-428f-b6fb-1f34c974bac7 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:30.069191 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1232/2489] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:30.278442 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edd0e3f-a6d9-4b47-ab91-4f1dd6a4f4b7 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:30.320694 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:30.344416 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edd0e3f-a6d9-4b47-ab91-4f1dd6a4f4b7 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:30.348887 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed72bd-833d-4803-8f54-c1871996cf24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:30.349502 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed72bd-833d-4803-8f54-c1871996cf24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:30.349777 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed72bd-833d-4803-8f54-c1871996cf24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:30.350047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed72bd-833d-4803-8f54-c1871996cf24 admin admin] PATH_INFO: `/v3/users/6c5ce5557d60473f9b85373892ad5c8f` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:30.351191 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ed72bd-833d-4803-8f54-c1871996cf24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c5ce5557d60473f9b85373892ad5c8f)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:30.353087 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ed72bd-833d-4803-8f54-c1871996cf24 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:30.398187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b2ed72bd-833d-4803-8f54-c1871996cf24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c5ce5557d60473f9b85373892ad5c8f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:30.406084 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b2ed72bd-833d-4803-8f54-c1871996cf24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c5ce5557d60473f9b85373892ad5c8f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:30.406759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b2ed72bd-833d-4803-8f54-c1871996cf24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c5ce5557d60473f9b85373892ad5c8f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:30.407357 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b2ed72bd-833d-4803-8f54-c1871996cf24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c5ce5557d60473f9b85373892ad5c8f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:30.426752 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b2ed72bd-833d-4803-8f54-c1871996cf24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c5ce5557d60473f9b85373892ad5c8f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:30.440144 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1258/2490] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:42:30 2022] DELETE /identity/v3/users/6c5ce5557d60473f9b85373892ad5c8f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:30.444815 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15833608-ac0a-4760-90be-b91c9631f7c3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:30.445120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15833608-ac0a-4760-90be-b91c9631f7c3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:30.445396 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15833608-ac0a-4760-90be-b91c9631f7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:30.464907 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-15833608-ac0a-4760-90be-b91c9631f7c3 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:30.466158 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-15833608-ac0a-4760-90be-b91c9631f7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:30.502488 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:30.532888 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1233/2491] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:30.543473 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed72bd-833d-4803-8f54-c1871996cf24 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:30.544859 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b2ed72bd-833d-4803-8f54-c1871996cf24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:30.607648 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed72bd-833d-4803-8f54-c1871996cf24 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:30.611231 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c7071e-7ecd-4bc0-b5c0-ab30453ec77f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:30.611827 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7071e-7ecd-4bc0-b5c0-ab30453ec77f service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:30.612106 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7071e-7ecd-4bc0-b5c0-ab30453ec77f service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:30.612378 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7071e-7ecd-4bc0-b5c0-ab30453ec77f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:30.621879 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c7071e-7ecd-4bc0-b5c0-ab30453ec77f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:30.623342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c7071e-7ecd-4bc0-b5c0-ab30453ec77f service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:30.632784 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1259/2492] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:30.912973 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15833608-ac0a-4760-90be-b91c9631f7c3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:30.914750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-15833608-ac0a-4760-90be-b91c9631f7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:30.951307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15833608-ac0a-4760-90be-b91c9631f7c3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:30.956051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98270664-c611-464a-bfa5-be8d42580c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:30.956713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98270664-c611-464a-bfa5-be8d42580c14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:30.956992 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98270664-c611-464a-bfa5-be8d42580c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:30.957263 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98270664-c611-464a-bfa5-be8d42580c14 admin admin] PATH_INFO: `/v3/projects/c38be99d56404657938612fa8c06999f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:30.958400 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98270664-c611-464a-bfa5-be8d42580c14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c38be99d56404657938612fa8c06999f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:30.960301 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98270664-c611-464a-bfa5-be8d42580c14 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:31.013627 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-98270664-c611-464a-bfa5-be8d42580c14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c38be99d56404657938612fa8c06999f', 'request_id': 'req-98270664-c611-464a-bfa5-be8d42580c14'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:31.019969 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-98270664-c611-464a-bfa5-be8d42580c14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c38be99d56404657938612fa8c06999f', 'request_id': 'req-98270664-c611-464a-bfa5-be8d42580c14'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:31.028128 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-98270664-c611-464a-bfa5-be8d42580c14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c38be99d56404657938612fa8c06999f', 'request_id': 'req-98270664-c611-464a-bfa5-be8d42580c14'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:31.038731 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-98270664-c611-464a-bfa5-be8d42580c14 admin admin] The token cache is being invalidate because project c38be99d56404657938612fa8c06999f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:42:31.039020 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-98270664-c611-464a-bfa5-be8d42580c14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:31.039639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-98270664-c611-464a-bfa5-be8d42580c14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:31.041158 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1234/2493] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:42:30 2022] DELETE /identity/v3/projects/c38be99d56404657938612fa8c06999f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:31.046652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7eeea6-5af0-4650-9e16-c4ab1c75e635 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:31.046932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7eeea6-5af0-4650-9e16-c4ab1c75e635 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:31.047228 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7eeea6-5af0-4650-9e16-c4ab1c75e635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:31.069925 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-af7eeea6-5af0-4650-9e16-c4ab1c75e635 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:31.071181 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-af7eeea6-5af0-4650-9e16-c4ab1c75e635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:31.132002 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:31.135912 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1260/2494] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:42:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:31.144113 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc0490-2e9a-44a8-9568-ea81b68606b3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:31.144397 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc0490-2e9a-44a8-9568-ea81b68606b3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:31.144671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc0490-2e9a-44a8-9568-ea81b68606b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:31.164278 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e4bc0490-2e9a-44a8-9568-ea81b68606b3 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:31.165469 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e4bc0490-2e9a-44a8-9568-ea81b68606b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:31.174368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1235/2495] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:42:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:31.178786 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7eeea6-5af0-4650-9e16-c4ab1c75e635 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:31.190217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7eeea6-5af0-4650-9e16-c4ab1c75e635 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:31.195147 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2a378-0f8d-4156-a8b1-f000287cf16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:31.195753 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2a378-0f8d-4156-a8b1-f000287cf16b admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:31.196033 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2a378-0f8d-4156-a8b1-f000287cf16b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:31.196304 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2a378-0f8d-4156-a8b1-f000287cf16b admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:31.196968 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef2a378-0f8d-4156-a8b1-f000287cf16b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:42:31.197410 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef2a378-0f8d-4156-a8b1-f000287cf16b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:31.199108 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef2a378-0f8d-4156-a8b1-f000287cf16b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:31.214156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1261/2496] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:42:31.215325 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc0490-2e9a-44a8-9568-ea81b68606b3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:31.226597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc0490-2e9a-44a8-9568-ea81b68606b3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:31.231891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8066ae-5fff-46a9-af8b-8c070ff024a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:31.232554 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8066ae-5fff-46a9-af8b-8c070ff024a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:31.232873 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8066ae-5fff-46a9-af8b-8c070ff024a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:31.233174 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8066ae-5fff-46a9-af8b-8c070ff024a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:31.233828 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8066ae-5fff-46a9-af8b-8c070ff024a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:31.235763 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8066ae-5fff-46a9-af8b-8c070ff024a1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:31.267449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1236/2497] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:42:31 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:31.271439 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2a378-0f8d-4156-a8b1-f000287cf16b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:31.282506 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2a378-0f8d-4156-a8b1-f000287cf16b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:31.345684 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999eab1e-3380-4acd-a7af-387ab7157df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:31.346452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999eab1e-3380-4acd-a7af-387ab7157df5 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:31.346735 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999eab1e-3380-4acd-a7af-387ab7157df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:31.347003 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999eab1e-3380-4acd-a7af-387ab7157df5 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:31.347643 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999eab1e-3380-4acd-a7af-387ab7157df5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:31.350251 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999eab1e-3380-4acd-a7af-387ab7157df5 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:31.373053 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1262/2498] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:42:31 2022] POST /identity/v3/users => generated 283 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:31.377133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8066ae-5fff-46a9-af8b-8c070ff024a1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:31.388302 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8066ae-5fff-46a9-af8b-8c070ff024a1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:31.393440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975837e2-cf2a-44a2-a080-f6b0fc8c545c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:31.394351 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975837e2-cf2a-44a2-a080-f6b0fc8c545c admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:31.394638 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975837e2-cf2a-44a2-a080-f6b0fc8c545c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:31.394914 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975837e2-cf2a-44a2-a080-f6b0fc8c545c admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:31.395561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975837e2-cf2a-44a2-a080-f6b0fc8c545c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:31.397167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975837e2-cf2a-44a2-a080-f6b0fc8c545c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:31.408239 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1237/2499] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:31.412302 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999eab1e-3380-4acd-a7af-387ab7157df5 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:31.422939 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999eab1e-3380-4acd-a7af-387ab7157df5 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:31.427509 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f20a4-98cd-47f1-8e50-d0f41d5ace7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:31.428099 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f20a4-98cd-47f1-8e50-d0f41d5ace7c admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:31.428380 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f20a4-98cd-47f1-8e50-d0f41d5ace7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:31.428654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f20a4-98cd-47f1-8e50-d0f41d5ace7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:31.429331 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f20a4-98cd-47f1-8e50-d0f41d5ace7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:31.431118 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f20a4-98cd-47f1-8e50-d0f41d5ace7c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:31.442751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1263/2500] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:31.446810 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975837e2-cf2a-44a2-a080-f6b0fc8c545c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:31.456818 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975837e2-cf2a-44a2-a080-f6b0fc8c545c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:31.461408 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f02251a-3436-4ba4-9f25-ff055440e065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:31.462060 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02251a-3436-4ba4-9f25-ff055440e065 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:31.462382 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02251a-3436-4ba4-9f25-ff055440e065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:31.462665 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02251a-3436-4ba4-9f25-ff055440e065 admin admin] PATH_INFO: `/v3/projects/0a2559c4ef434ff9917cef63927a0728/users/c4855937c6f642f3a938f989a9868be6/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:31.479540 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f02251a-3436-4ba4-9f25-ff055440e065 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a2559c4ef434ff9917cef63927a0728, user_id=c4855937c6f642f3a938f989a9868be6, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:31.482978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f02251a-3436-4ba4-9f25-ff055440e065 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:31.496388 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1238/2501] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:31 2022] PUT /identity/v3/projects/0a2559c4ef434ff9917cef63927a0728/users/c4855937c6f642f3a938f989a9868be6/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:31.507981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a9f33-91f3-40ec-8727-8f4e0d08368a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:31.508273 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a9f33-91f3-40ec-8727-8f4e0d08368a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:31.508545 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a9f33-91f3-40ec-8727-8f4e0d08368a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:31.535964 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-fb5a9f33-91f3-40ec-8727-8f4e0d08368a None None] MFA Rules not processed for user `c4855937c6f642f3a938f989a9868be6`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:31.537168 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-fb5a9f33-91f3-40ec-8727-8f4e0d08368a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:31.600666 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1264/2502] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:42:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:31.610943 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f02251a-3436-4ba4-9f25-ff055440e065 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:31.671612 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:31.677930 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f02251a-3436-4ba4-9f25-ff055440e065 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:31.683788 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96828b91-0216-42f9-8cda-94b7c8ab9ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:31.684396 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96828b91-0216-42f9-8cda-94b7c8ab9ca4 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:31.684709 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96828b91-0216-42f9-8cda-94b7c8ab9ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:31.684954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96828b91-0216-42f9-8cda-94b7c8ab9ca4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:31.685809 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96828b91-0216-42f9-8cda-94b7c8ab9ca4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:31.688147 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96828b91-0216-42f9-8cda-94b7c8ab9ca4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:31.719870 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1239/2503] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:42:31 2022] POST /identity/v3/projects => generated 365 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:31.723705 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a9f33-91f3-40ec-8727-8f4e0d08368a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:31.734654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a9f33-91f3-40ec-8727-8f4e0d08368a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:31.784094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:31.792130 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c869e-d218-4d86-9bd3-b8fdb4309b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:31.792775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c869e-d218-4d86-9bd3-b8fdb4309b54 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:31.793062 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c869e-d218-4d86-9bd3-b8fdb4309b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:31.793337 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c869e-d218-4d86-9bd3-b8fdb4309b54 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:31.793969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c869e-d218-4d86-9bd3-b8fdb4309b54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:31.795938 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c869e-d218-4d86-9bd3-b8fdb4309b54 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:31.817580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1265/2504] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:42:31 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:31.823386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96828b91-0216-42f9-8cda-94b7c8ab9ca4 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:31.841604 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96828b91-0216-42f9-8cda-94b7c8ab9ca4 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:31.850296 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2de2af-fa56-4f25-a73d-97b3e00c71a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:31.850915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2de2af-fa56-4f25-a73d-97b3e00c71a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:31.851549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2de2af-fa56-4f25-a73d-97b3e00c71a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:31.852179 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2de2af-fa56-4f25-a73d-97b3e00c71a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:31.852831 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2de2af-fa56-4f25-a73d-97b3e00c71a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:31.854471 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2de2af-fa56-4f25-a73d-97b3e00c71a2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:31.866919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1240/2505] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:31.870963 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c869e-d218-4d86-9bd3-b8fdb4309b54 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:31.881526 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c869e-d218-4d86-9bd3-b8fdb4309b54 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:31.886187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d15c34-5249-40ff-8688-4b9627c572f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:31.887358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d15c34-5249-40ff-8688-4b9627c572f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:31.887636 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d15c34-5249-40ff-8688-4b9627c572f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:31.887947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d15c34-5249-40ff-8688-4b9627c572f0 admin admin] PATH_INFO: `/v3/projects/cb0193900b2540c18ba01f648c012cf0/users/cfdb9a831c9148a2b9993fa5b9eaec47/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:31.910305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d15c34-5249-40ff-8688-4b9627c572f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb0193900b2540c18ba01f648c012cf0, user_id=cfdb9a831c9148a2b9993fa5b9eaec47, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:31.913394 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d15c34-5249-40ff-8688-4b9627c572f0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:31.928271 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1266/2506] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:31 2022] PUT /identity/v3/projects/cb0193900b2540c18ba01f648c012cf0/users/cfdb9a831c9148a2b9993fa5b9eaec47/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:31.932203 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2de2af-fa56-4f25-a73d-97b3e00c71a2 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:31.942329 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2de2af-fa56-4f25-a73d-97b3e00c71a2 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:32.000041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92228c05-3e0a-4ea0-8bc0-2d178e6d4ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:32.000650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92228c05-3e0a-4ea0-8bc0-2d178e6d4ecc admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:32.000935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92228c05-3e0a-4ea0-8bc0-2d178e6d4ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:32.001214 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92228c05-3e0a-4ea0-8bc0-2d178e6d4ecc admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:32.001883 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92228c05-3e0a-4ea0-8bc0-2d178e6d4ecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:32.003530 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92228c05-3e0a-4ea0-8bc0-2d178e6d4ecc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:32.013947 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1241/2507] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:32.018347 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d15c34-5249-40ff-8688-4b9627c572f0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:32.028983 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d15c34-5249-40ff-8688-4b9627c572f0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:32.033672 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6e57e-bacc-4ddb-bc67-79d0ac9d4ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:32.034372 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6e57e-bacc-4ddb-bc67-79d0ac9d4ea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:32.034656 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6e57e-bacc-4ddb-bc67-79d0ac9d4ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:32.034932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6e57e-bacc-4ddb-bc67-79d0ac9d4ea2 admin admin] PATH_INFO: `/v3/projects/cb0193900b2540c18ba01f648c012cf0/users/cfdb9a831c9148a2b9993fa5b9eaec47/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:32.036994 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d6e57e-bacc-4ddb-bc67-79d0ac9d4ea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb0193900b2540c18ba01f648c012cf0, user_id=cfdb9a831c9148a2b9993fa5b9eaec47, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:32.039664 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d6e57e-bacc-4ddb-bc67-79d0ac9d4ea2 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:32.050629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1267/2508] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:32 2022] PUT /identity/v3/projects/cb0193900b2540c18ba01f648c012cf0/users/cfdb9a831c9148a2b9993fa5b9eaec47/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:32.061783 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de325346-503b-4827-85fd-1a1018599f66 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:32.062075 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de325346-503b-4827-85fd-1a1018599f66 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:32.062397 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de325346-503b-4827-85fd-1a1018599f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:32.090423 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-de325346-503b-4827-85fd-1a1018599f66 None None] MFA Rules not processed for user `cfdb9a831c9148a2b9993fa5b9eaec47`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:32.091615 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-de325346-503b-4827-85fd-1a1018599f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:32.162431 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1242/2509] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:42:32 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:32.173814 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6e57e-bacc-4ddb-bc67-79d0ac9d4ea2 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:32.175540 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b5d6e57e-bacc-4ddb-bc67-79d0ac9d4ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:32.232014 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6e57e-bacc-4ddb-bc67-79d0ac9d4ea2 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:32.235615 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d3634-5710-4a57-b7a5-f12480259a28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:32.236217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d3634-5710-4a57-b7a5-f12480259a28 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:32.236500 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d3634-5710-4a57-b7a5-f12480259a28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:32.236777 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d3634-5710-4a57-b7a5-f12480259a28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:32.245563 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d3634-5710-4a57-b7a5-f12480259a28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:32.247509 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d3634-5710-4a57-b7a5-f12480259a28 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:32.256733 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1268/2510] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:32 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:32.669810 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de325346-503b-4827-85fd-1a1018599f66 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:32.720675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:32.734953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de325346-503b-4827-85fd-1a1018599f66 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:32.739422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b742e-9dc6-4e1c-a2fe-a6cfa5537664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:32.740046 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b742e-9dc6-4e1c-a2fe-a6cfa5537664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:32.740326 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b742e-9dc6-4e1c-a2fe-a6cfa5537664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:32.740596 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b742e-9dc6-4e1c-a2fe-a6cfa5537664 admin admin] PATH_INFO: `/v3/users/c4855937c6f642f3a938f989a9868be6` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:32.741727 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b742e-9dc6-4e1c-a2fe-a6cfa5537664 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4855937c6f642f3a938f989a9868be6)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:32.743657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b742e-9dc6-4e1c-a2fe-a6cfa5537664 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:32.790715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-788b742e-9dc6-4e1c-a2fe-a6cfa5537664 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4855937c6f642f3a938f989a9868be6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:32.798050 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-788b742e-9dc6-4e1c-a2fe-a6cfa5537664 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4855937c6f642f3a938f989a9868be6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:32.798727 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-788b742e-9dc6-4e1c-a2fe-a6cfa5537664 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4855937c6f642f3a938f989a9868be6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:32.805728 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-788b742e-9dc6-4e1c-a2fe-a6cfa5537664 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4855937c6f642f3a938f989a9868be6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:32.817551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-788b742e-9dc6-4e1c-a2fe-a6cfa5537664 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4855937c6f642f3a938f989a9868be6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:32.823377 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1243/2511] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:42:32 2022] DELETE /identity/v3/users/c4855937c6f642f3a938f989a9868be6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:32.827867 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf114b60-b04d-4db0-839e-8f82c3afb958 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:32.828387 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf114b60-b04d-4db0-839e-8f82c3afb958 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:32.828495 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf114b60-b04d-4db0-839e-8f82c3afb958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:32.847391 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-bf114b60-b04d-4db0-839e-8f82c3afb958 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:32.848585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bf114b60-b04d-4db0-839e-8f82c3afb958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:32.909212 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1269/2512] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:32.919388 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b742e-9dc6-4e1c-a2fe-a6cfa5537664 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:32.920950 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-788b742e-9dc6-4e1c-a2fe-a6cfa5537664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:32.976688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b742e-9dc6-4e1c-a2fe-a6cfa5537664 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:32.980272 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b6868-7a7c-44be-875d-7585090c8d9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:32.981407 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b6868-7a7c-44be-875d-7585090c8d9d service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:32.981688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b6868-7a7c-44be-875d-7585090c8d9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:32.981958 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b6868-7a7c-44be-875d-7585090c8d9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:32.990761 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8b6868-7a7c-44be-875d-7585090c8d9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:32.992173 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8b6868-7a7c-44be-875d-7585090c8d9d service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:33.001467 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1244/2513] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:33.270500 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf114b60-b04d-4db0-839e-8f82c3afb958 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:33.271917 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bf114b60-b04d-4db0-839e-8f82c3afb958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:33.286003 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf114b60-b04d-4db0-839e-8f82c3afb958 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:33.290963 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc09d6-ef72-4318-9d37-30d2bd6c98c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:33.291306 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc09d6-ef72-4318-9d37-30d2bd6c98c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:33.291610 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc09d6-ef72-4318-9d37-30d2bd6c98c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:33.291881 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc09d6-ef72-4318-9d37-30d2bd6c98c9 admin admin] PATH_INFO: `/v3/projects/0a2559c4ef434ff9917cef63927a0728` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:33.293015 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc09d6-ef72-4318-9d37-30d2bd6c98c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a2559c4ef434ff9917cef63927a0728)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:33.294956 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc09d6-ef72-4318-9d37-30d2bd6c98c9 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:33.339606 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:33.343303 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c0cc09d6-ef72-4318-9d37-30d2bd6c98c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a2559c4ef434ff9917cef63927a0728', 'request_id': 'req-c0cc09d6-ef72-4318-9d37-30d2bd6c98c9'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:33.351911 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c0cc09d6-ef72-4318-9d37-30d2bd6c98c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a2559c4ef434ff9917cef63927a0728', 'request_id': 'req-c0cc09d6-ef72-4318-9d37-30d2bd6c98c9'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:33.357509 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c0cc09d6-ef72-4318-9d37-30d2bd6c98c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a2559c4ef434ff9917cef63927a0728', 'request_id': 'req-c0cc09d6-ef72-4318-9d37-30d2bd6c98c9'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:33.370417 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c0cc09d6-ef72-4318-9d37-30d2bd6c98c9 admin admin] The token cache is being invalidate because project 0a2559c4ef434ff9917cef63927a0728 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:42:33.370704 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c0cc09d6-ef72-4318-9d37-30d2bd6c98c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:33.371411 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c0cc09d6-ef72-4318-9d37-30d2bd6c98c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:33.372871 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1270/2514] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:42:33 2022] DELETE /identity/v3/projects/0a2559c4ef434ff9917cef63927a0728 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:33.376960 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b6868-7a7c-44be-875d-7585090c8d9d service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:33.387793 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b6868-7a7c-44be-875d-7585090c8d9d service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:33.444096 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9775691e-dc2e-4e4a-98b1-85a1a74ee186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:33.444722 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775691e-dc2e-4e4a-98b1-85a1a74ee186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:33.445007 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775691e-dc2e-4e4a-98b1-85a1a74ee186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:33.445288 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775691e-dc2e-4e4a-98b1-85a1a74ee186 admin admin] PATH_INFO: `/v3/users/cfdb9a831c9148a2b9993fa5b9eaec47` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:33.446440 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9775691e-dc2e-4e4a-98b1-85a1a74ee186 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfdb9a831c9148a2b9993fa5b9eaec47)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:33.448339 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9775691e-dc2e-4e4a-98b1-85a1a74ee186 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:33.461948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:33.495815 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9775691e-dc2e-4e4a-98b1-85a1a74ee186 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfdb9a831c9148a2b9993fa5b9eaec47'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:33.503108 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9775691e-dc2e-4e4a-98b1-85a1a74ee186 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfdb9a831c9148a2b9993fa5b9eaec47'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:33.503767 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9775691e-dc2e-4e4a-98b1-85a1a74ee186 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfdb9a831c9148a2b9993fa5b9eaec47'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:33.511390 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9775691e-dc2e-4e4a-98b1-85a1a74ee186 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfdb9a831c9148a2b9993fa5b9eaec47'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:33.523942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9775691e-dc2e-4e4a-98b1-85a1a74ee186 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfdb9a831c9148a2b9993fa5b9eaec47'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:33.530479 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1245/2515] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:42:33 2022] DELETE /identity/v3/users/cfdb9a831c9148a2b9993fa5b9eaec47 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:33.799617 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc09d6-ef72-4318-9d37-30d2bd6c98c9 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:33.801197 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c0cc09d6-ef72-4318-9d37-30d2bd6c98c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:33.815255 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc09d6-ef72-4318-9d37-30d2bd6c98c9 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:33.874024 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b11ef4-7ca1-481a-be62-4054d209cd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:33.874719 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b11ef4-7ca1-481a-be62-4054d209cd3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:33.874969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b11ef4-7ca1-481a-be62-4054d209cd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:33.875245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b11ef4-7ca1-481a-be62-4054d209cd3e admin admin] PATH_INFO: `/v3/projects/cb0193900b2540c18ba01f648c012cf0` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:33.876362 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b11ef4-7ca1-481a-be62-4054d209cd3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb0193900b2540c18ba01f648c012cf0)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:33.878377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b11ef4-7ca1-481a-be62-4054d209cd3e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:33.924278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f4b11ef4-7ca1-481a-be62-4054d209cd3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb0193900b2540c18ba01f648c012cf0', 'request_id': 'req-f4b11ef4-7ca1-481a-be62-4054d209cd3e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:33.930416 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f4b11ef4-7ca1-481a-be62-4054d209cd3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb0193900b2540c18ba01f648c012cf0', 'request_id': 'req-f4b11ef4-7ca1-481a-be62-4054d209cd3e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:33.936396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f4b11ef4-7ca1-481a-be62-4054d209cd3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb0193900b2540c18ba01f648c012cf0', 'request_id': 'req-f4b11ef4-7ca1-481a-be62-4054d209cd3e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:33.948365 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f4b11ef4-7ca1-481a-be62-4054d209cd3e admin admin] The token cache is being invalidate because project cb0193900b2540c18ba01f648c012cf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:42:33.948656 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f4b11ef4-7ca1-481a-be62-4054d209cd3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:33.949355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f4b11ef4-7ca1-481a-be62-4054d209cd3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:33.950890 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1271/2516] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:42:33 2022] DELETE /identity/v3/projects/cb0193900b2540c18ba01f648c012cf0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:33.956825 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce49e00-5b97-40a9-a85a-02c6608f7f3e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:33.957114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce49e00-5b97-40a9-a85a-02c6608f7f3e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:33.957390 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce49e00-5b97-40a9-a85a-02c6608f7f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:33.977047 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4ce49e00-5b97-40a9-a85a-02c6608f7f3e None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:33.978309 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4ce49e00-5b97-40a9-a85a-02c6608f7f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:34.037443 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1246/2517] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:34.047281 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34279f2-8ccc-45fb-b4c8-e00d12b1df39 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:34.047565 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34279f2-8ccc-45fb-b4c8-e00d12b1df39 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:34.047858 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34279f2-8ccc-45fb-b4c8-e00d12b1df39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:34.068029 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e34279f2-8ccc-45fb-b4c8-e00d12b1df39 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:34.069229 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e34279f2-8ccc-45fb-b4c8-e00d12b1df39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:34.077116 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1272/2518] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:34.081343 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce49e00-5b97-40a9-a85a-02c6608f7f3e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:34.090933 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce49e00-5b97-40a9-a85a-02c6608f7f3e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:34.095457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c623f3d1-63e0-461d-ae2c-daad81cf6163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:34.096052 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623f3d1-63e0-461d-ae2c-daad81cf6163 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:34.096334 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623f3d1-63e0-461d-ae2c-daad81cf6163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:34.096606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623f3d1-63e0-461d-ae2c-daad81cf6163 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:34.097248 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c623f3d1-63e0-461d-ae2c-daad81cf6163 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:42:34.097551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c623f3d1-63e0-461d-ae2c-daad81cf6163 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:34.099166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c623f3d1-63e0-461d-ae2c-daad81cf6163 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:34.110863 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1247/2519] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:42:34.115141 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34279f2-8ccc-45fb-b4c8-e00d12b1df39 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:34.124932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34279f2-8ccc-45fb-b4c8-e00d12b1df39 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:34.129535 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13968462-ff76-4dfb-a616-96ea97f46a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:34.130084 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13968462-ff76-4dfb-a616-96ea97f46a6d admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:34.130385 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13968462-ff76-4dfb-a616-96ea97f46a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:34.130656 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13968462-ff76-4dfb-a616-96ea97f46a6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:34.131283 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13968462-ff76-4dfb-a616-96ea97f46a6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:34.133201 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13968462-ff76-4dfb-a616-96ea97f46a6d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:34.160183 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1273/2520] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:42:34 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:34.164446 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c623f3d1-63e0-461d-ae2c-daad81cf6163 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:34.174716 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c623f3d1-63e0-461d-ae2c-daad81cf6163 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:34.229408 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8f55a-787d-43c3-868e-5c7a7e07ee83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:34.230000 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8f55a-787d-43c3-868e-5c7a7e07ee83 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:34.230314 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8f55a-787d-43c3-868e-5c7a7e07ee83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:34.231124 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8f55a-787d-43c3-868e-5c7a7e07ee83 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:34.231739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8f55a-787d-43c3-868e-5c7a7e07ee83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:34.233659 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8f55a-787d-43c3-868e-5c7a7e07ee83 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:34.254736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1248/2521] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:42:34 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:34.258791 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13968462-ff76-4dfb-a616-96ea97f46a6d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:34.268970 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13968462-ff76-4dfb-a616-96ea97f46a6d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:34.273588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364abcc-188d-4db6-ab69-0356de6add78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:34.274178 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364abcc-188d-4db6-ab69-0356de6add78 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:34.274489 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364abcc-188d-4db6-ab69-0356de6add78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:34.274759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364abcc-188d-4db6-ab69-0356de6add78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:34.275651 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364abcc-188d-4db6-ab69-0356de6add78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:34.277226 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364abcc-188d-4db6-ab69-0356de6add78 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:34.287645 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1274/2522] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:34.291549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8f55a-787d-43c3-868e-5c7a7e07ee83 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:34.302099 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8f55a-787d-43c3-868e-5c7a7e07ee83 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:34.306906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36084f8-b2e0-4655-9181-1cbfd1ffc34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:34.307495 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36084f8-b2e0-4655-9181-1cbfd1ffc34c admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:34.307780 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36084f8-b2e0-4655-9181-1cbfd1ffc34c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:34.308057 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36084f8-b2e0-4655-9181-1cbfd1ffc34c admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:34.308727 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36084f8-b2e0-4655-9181-1cbfd1ffc34c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:34.310360 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36084f8-b2e0-4655-9181-1cbfd1ffc34c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:34.312442 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:34.322177 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1249/2523] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:34.327861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364abcc-188d-4db6-ab69-0356de6add78 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:34.337630 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364abcc-188d-4db6-ab69-0356de6add78 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:34.342693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8441c343-fba9-4b52-ab6c-5daf7fb7178c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:34.343325 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441c343-fba9-4b52-ab6c-5daf7fb7178c admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:34.343601 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441c343-fba9-4b52-ab6c-5daf7fb7178c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:34.343873 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441c343-fba9-4b52-ab6c-5daf7fb7178c admin admin] PATH_INFO: `/v3/projects/b9dccbe731b241ee90efda875f0c7610/users/a6537006ef7c47b2be92ee074a0d7e92/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:34.359223 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8441c343-fba9-4b52-ab6c-5daf7fb7178c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9dccbe731b241ee90efda875f0c7610, user_id=a6537006ef7c47b2be92ee074a0d7e92, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:34.362303 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8441c343-fba9-4b52-ab6c-5daf7fb7178c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:34.371320 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:34.376548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1275/2524] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:34 2022] PUT /identity/v3/projects/b9dccbe731b241ee90efda875f0c7610/users/a6537006ef7c47b2be92ee074a0d7e92/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:34.387363 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aaf616-8773-4f12-8f60-d39648769d78 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:34.387851 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aaf616-8773-4f12-8f60-d39648769d78 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:34.387851 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aaf616-8773-4f12-8f60-d39648769d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:34.415579 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b6aaf616-8773-4f12-8f60-d39648769d78 None None] MFA Rules not processed for user `a6537006ef7c47b2be92ee074a0d7e92`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:34.416788 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b6aaf616-8773-4f12-8f60-d39648769d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:34.481778 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1250/2525] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:42:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:34.486137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8441c343-fba9-4b52-ab6c-5daf7fb7178c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:34.550691 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8441c343-fba9-4b52-ab6c-5daf7fb7178c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:34.556265 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1d37b-0406-4c18-a6d5-880a06c1930c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:34.557303 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1d37b-0406-4c18-a6d5-880a06c1930c admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:34.557769 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1d37b-0406-4c18-a6d5-880a06c1930c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:34.558323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1d37b-0406-4c18-a6d5-880a06c1930c admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:34.559508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f1d37b-0406-4c18-a6d5-880a06c1930c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:34.562847 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f1d37b-0406-4c18-a6d5-880a06c1930c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:34.592154 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1276/2526] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:42:34 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:34.596193 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aaf616-8773-4f12-8f60-d39648769d78 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:34.607297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aaf616-8773-4f12-8f60-d39648769d78 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:34.664583 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15da49-ac87-4c7f-9bdd-4a60278a1eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:34.665190 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15da49-ac87-4c7f-9bdd-4a60278a1eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:34.665475 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15da49-ac87-4c7f-9bdd-4a60278a1eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:34.665751 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15da49-ac87-4c7f-9bdd-4a60278a1eb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:34.666454 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d15da49-ac87-4c7f-9bdd-4a60278a1eb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:34.668363 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d15da49-ac87-4c7f-9bdd-4a60278a1eb9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:34.691855 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1251/2527] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:42:34 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:34.693288 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1d37b-0406-4c18-a6d5-880a06c1930c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:34.704136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1d37b-0406-4c18-a6d5-880a06c1930c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:34.709699 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f5785-c765-4fb7-b4bd-a60866c4008f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:34.710354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f5785-c765-4fb7-b4bd-a60866c4008f admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:34.710636 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f5785-c765-4fb7-b4bd-a60866c4008f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:34.710907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f5785-c765-4fb7-b4bd-a60866c4008f admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:34.711549 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f5785-c765-4fb7-b4bd-a60866c4008f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:34.713129 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f5785-c765-4fb7-b4bd-a60866c4008f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:34.724590 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1277/2528] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:34.728747 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15da49-ac87-4c7f-9bdd-4a60278a1eb9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:34.739004 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15da49-ac87-4c7f-9bdd-4a60278a1eb9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:34.743660 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39112aed-243a-48f3-b61a-5ab650b47995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:34.744306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39112aed-243a-48f3-b61a-5ab650b47995 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:34.744592 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39112aed-243a-48f3-b61a-5ab650b47995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:34.744868 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39112aed-243a-48f3-b61a-5ab650b47995 admin admin] PATH_INFO: `/v3/projects/a87c4efb583945d797cca62eaec29f97/users/f2be42dd60a24f46a5dfa5d3564fbab7/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:34.759627 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39112aed-243a-48f3-b61a-5ab650b47995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a87c4efb583945d797cca62eaec29f97, user_id=f2be42dd60a24f46a5dfa5d3564fbab7, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:34.762863 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39112aed-243a-48f3-b61a-5ab650b47995 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:34.776098 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1252/2529] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:34 2022] PUT /identity/v3/projects/a87c4efb583945d797cca62eaec29f97/users/f2be42dd60a24f46a5dfa5d3564fbab7/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:34.780669 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f5785-c765-4fb7-b4bd-a60866c4008f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:34.790786 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f5785-c765-4fb7-b4bd-a60866c4008f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:34.845847 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca0741-83d1-4c80-8783-c2a488cd0323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:34.846467 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca0741-83d1-4c80-8783-c2a488cd0323 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:34.846747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca0741-83d1-4c80-8783-c2a488cd0323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:34.847018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca0741-83d1-4c80-8783-c2a488cd0323 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:34.847655 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca0741-83d1-4c80-8783-c2a488cd0323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:34.849232 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca0741-83d1-4c80-8783-c2a488cd0323 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:34.858691 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1278/2530] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:42:34.863061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39112aed-243a-48f3-b61a-5ab650b47995 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:34.873977 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39112aed-243a-48f3-b61a-5ab650b47995 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:34.878845 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b9157-5c72-4bd6-94f2-ce4d5c7032df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:34.879540 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b9157-5c72-4bd6-94f2-ce4d5c7032df admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:34.879826 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b9157-5c72-4bd6-94f2-ce4d5c7032df admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:34.880127 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b9157-5c72-4bd6-94f2-ce4d5c7032df admin admin] PATH_INFO: `/v3/projects/a87c4efb583945d797cca62eaec29f97/users/f2be42dd60a24f46a5dfa5d3564fbab7/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:34.882162 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b9157-5c72-4bd6-94f2-ce4d5c7032df admin admin] RBAC: Authorizing `identity:create_grant(project_id=a87c4efb583945d797cca62eaec29f97, user_id=f2be42dd60a24f46a5dfa5d3564fbab7, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:34.885171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b9157-5c72-4bd6-94f2-ce4d5c7032df admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:34.896449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1253/2531] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:34 2022] PUT /identity/v3/projects/a87c4efb583945d797cca62eaec29f97/users/f2be42dd60a24f46a5dfa5d3564fbab7/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:34.907954 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788c985-2ae2-469d-b90f-86adecc3cccc None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:34.908555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788c985-2ae2-469d-b90f-86adecc3cccc None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:34.908555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788c985-2ae2-469d-b90f-86adecc3cccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:34.936012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5788c985-2ae2-469d-b90f-86adecc3cccc None None] MFA Rules not processed for user `f2be42dd60a24f46a5dfa5d3564fbab7`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:34.937127 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5788c985-2ae2-469d-b90f-86adecc3cccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:34.995616 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:35.006903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1279/2532] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:42:34 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:35.018147 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b9157-5c72-4bd6-94f2-ce4d5c7032df admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:35.019887 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bf1b9157-5c72-4bd6-94f2-ce4d5c7032df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:35.064961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:35.080322 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b9157-5c72-4bd6-94f2-ce4d5c7032df admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:35.084032 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724cc2b-2233-4cd2-848d-de81f2c2b976 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:35.084636 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724cc2b-2233-4cd2-848d-de81f2c2b976 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:35.084920 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724cc2b-2233-4cd2-848d-de81f2c2b976 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:35.085197 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724cc2b-2233-4cd2-848d-de81f2c2b976 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:35.094008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724cc2b-2233-4cd2-848d-de81f2c2b976 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:35.095479 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724cc2b-2233-4cd2-848d-de81f2c2b976 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:35.104794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1254/2533] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:35.465392 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788c985-2ae2-469d-b90f-86adecc3cccc None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:35.528947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788c985-2ae2-469d-b90f-86adecc3cccc None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:35.533707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb171003-f796-49f5-8f9e-bb0990f736c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:35.534396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb171003-f796-49f5-8f9e-bb0990f736c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:35.534708 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb171003-f796-49f5-8f9e-bb0990f736c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:35.534984 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb171003-f796-49f5-8f9e-bb0990f736c7 admin admin] PATH_INFO: `/v3/users/a6537006ef7c47b2be92ee074a0d7e92` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:35.536152 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb171003-f796-49f5-8f9e-bb0990f736c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6537006ef7c47b2be92ee074a0d7e92)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:35.538267 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb171003-f796-49f5-8f9e-bb0990f736c7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:35.585332 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bb171003-f796-49f5-8f9e-bb0990f736c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6537006ef7c47b2be92ee074a0d7e92'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:35.593052 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bb171003-f796-49f5-8f9e-bb0990f736c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6537006ef7c47b2be92ee074a0d7e92'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:35.593751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bb171003-f796-49f5-8f9e-bb0990f736c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6537006ef7c47b2be92ee074a0d7e92'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:35.594412 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bb171003-f796-49f5-8f9e-bb0990f736c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6537006ef7c47b2be92ee074a0d7e92'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:35.607040 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bb171003-f796-49f5-8f9e-bb0990f736c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6537006ef7c47b2be92ee074a0d7e92'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:35.619687 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1280/2534] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:42:35 2022] DELETE /identity/v3/users/a6537006ef7c47b2be92ee074a0d7e92 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:35.625299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e8416-105f-48fc-a999-b690d8a10c7d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:35.625804 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e8416-105f-48fc-a999-b690d8a10c7d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:35.625917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e8416-105f-48fc-a999-b690d8a10c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:35.645032 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d17e8416-105f-48fc-a999-b690d8a10c7d None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:35.646229 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d17e8416-105f-48fc-a999-b690d8a10c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:35.705198 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1255/2535] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:42:35.714853 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb171003-f796-49f5-8f9e-bb0990f736c7 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:35.716421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bb171003-f796-49f5-8f9e-bb0990f736c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:35.775068 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb171003-f796-49f5-8f9e-bb0990f736c7 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:35.779506 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17c37c-eb14-4885-bce8-d21cc950572a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:35.780126 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17c37c-eb14-4885-bce8-d21cc950572a service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:35.780426 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17c37c-eb14-4885-bce8-d21cc950572a service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:35.780703 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17c37c-eb14-4885-bce8-d21cc950572a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:35.790116 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17c37c-eb14-4885-bce8-d21cc950572a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:35.791583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17c37c-eb14-4885-bce8-d21cc950572a service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:35.801616 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1281/2536] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:36.063822 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e8416-105f-48fc-a999-b690d8a10c7d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:36.065576 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d17e8416-105f-48fc-a999-b690d8a10c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:36.078101 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e8416-105f-48fc-a999-b690d8a10c7d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:36.082981 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f1258-a867-4136-a54c-c630d8d5056c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:36.083790 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f1258-a867-4136-a54c-c630d8d5056c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:36.084038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f1258-a867-4136-a54c-c630d8d5056c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:36.084325 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f1258-a867-4136-a54c-c630d8d5056c admin admin] PATH_INFO: `/v3/projects/b9dccbe731b241ee90efda875f0c7610` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:36.085660 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f1258-a867-4136-a54c-c630d8d5056c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9dccbe731b241ee90efda875f0c7610)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:36.087801 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f1258-a867-4136-a54c-c630d8d5056c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:36.134745 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e50f1258-a867-4136-a54c-c630d8d5056c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9dccbe731b241ee90efda875f0c7610', 'request_id': 'req-e50f1258-a867-4136-a54c-c630d8d5056c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:36.141242 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e50f1258-a867-4136-a54c-c630d8d5056c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9dccbe731b241ee90efda875f0c7610', 'request_id': 'req-e50f1258-a867-4136-a54c-c630d8d5056c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:36.147872 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e50f1258-a867-4136-a54c-c630d8d5056c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9dccbe731b241ee90efda875f0c7610', 'request_id': 'req-e50f1258-a867-4136-a54c-c630d8d5056c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:36.158961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e50f1258-a867-4136-a54c-c630d8d5056c admin admin] The token cache is being invalidate because project b9dccbe731b241ee90efda875f0c7610 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:42:36.159250 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e50f1258-a867-4136-a54c-c630d8d5056c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:36.159967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e50f1258-a867-4136-a54c-c630d8d5056c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:36.161444 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1256/2537] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:42:36 2022] DELETE /identity/v3/projects/b9dccbe731b241ee90efda875f0c7610 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:42:36.165403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17c37c-eb14-4885-bce8-d21cc950572a service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:36.175450 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17c37c-eb14-4885-bce8-d21cc950572a service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:36.187827 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:36.232837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932f816-9e40-4b14-8081-91c2649c3944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:36.233460 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932f816-9e40-4b14-8081-91c2649c3944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:36.233739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932f816-9e40-4b14-8081-91c2649c3944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:36.234013 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932f816-9e40-4b14-8081-91c2649c3944 admin admin] PATH_INFO: `/v3/users/f2be42dd60a24f46a5dfa5d3564fbab7` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:36.235160 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932f816-9e40-4b14-8081-91c2649c3944 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2be42dd60a24f46a5dfa5d3564fbab7)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:36.237070 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932f816-9e40-4b14-8081-91c2649c3944 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:36.283393 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5932f816-9e40-4b14-8081-91c2649c3944 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2be42dd60a24f46a5dfa5d3564fbab7'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:36.290393 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5932f816-9e40-4b14-8081-91c2649c3944 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2be42dd60a24f46a5dfa5d3564fbab7'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:36.291067 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5932f816-9e40-4b14-8081-91c2649c3944 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2be42dd60a24f46a5dfa5d3564fbab7'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:36.291661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5932f816-9e40-4b14-8081-91c2649c3944 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2be42dd60a24f46a5dfa5d3564fbab7'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:36.303744 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5932f816-9e40-4b14-8081-91c2649c3944 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2be42dd60a24f46a5dfa5d3564fbab7'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:36.316924 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1282/2538] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:42:36 2022] DELETE /identity/v3/users/f2be42dd60a24f46a5dfa5d3564fbab7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:36.584464 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f1258-a867-4136-a54c-c630d8d5056c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:36.586402 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e50f1258-a867-4136-a54c-c630d8d5056c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:36.600170 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f1258-a867-4136-a54c-c630d8d5056c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:36.659152 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a2258-408d-4134-b149-335fc7a55bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:36.659817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a2258-408d-4134-b149-335fc7a55bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:36.660051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a2258-408d-4134-b149-335fc7a55bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:36.660325 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a2258-408d-4134-b149-335fc7a55bd4 admin admin] PATH_INFO: `/v3/projects/a87c4efb583945d797cca62eaec29f97` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:36.661435 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2a2258-408d-4134-b149-335fc7a55bd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a87c4efb583945d797cca62eaec29f97)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:36.663381 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2a2258-408d-4134-b149-335fc7a55bd4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:36.671452 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:36.711870 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-fb2a2258-408d-4134-b149-335fc7a55bd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a87c4efb583945d797cca62eaec29f97', 'request_id': 'req-fb2a2258-408d-4134-b149-335fc7a55bd4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:36.718423 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-fb2a2258-408d-4134-b149-335fc7a55bd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a87c4efb583945d797cca62eaec29f97', 'request_id': 'req-fb2a2258-408d-4134-b149-335fc7a55bd4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:36.725851 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-fb2a2258-408d-4134-b149-335fc7a55bd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a87c4efb583945d797cca62eaec29f97', 'request_id': 'req-fb2a2258-408d-4134-b149-335fc7a55bd4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:36.741163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-fb2a2258-408d-4134-b149-335fc7a55bd4 admin admin] The token cache is being invalidate because project a87c4efb583945d797cca62eaec29f97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:42:36.741453 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-fb2a2258-408d-4134-b149-335fc7a55bd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:36.742093 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-fb2a2258-408d-4134-b149-335fc7a55bd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:36.743740 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1257/2539] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:42:36 2022] DELETE /identity/v3/projects/a87c4efb583945d797cca62eaec29f97 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:36.748649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd90e08-e79f-41ba-92c7-e0b5452bb26a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:36.748932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd90e08-e79f-41ba-92c7-e0b5452bb26a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:36.749200 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd90e08-e79f-41ba-92c7-e0b5452bb26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:36.768344 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-4bd90e08-e79f-41ba-92c7-e0b5452bb26a None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:36.769559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4bd90e08-e79f-41ba-92c7-e0b5452bb26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:36.829241 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1283/2540] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:42:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:36.838235 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a35b68-a94b-4200-85e0-6437e7fa8923 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:36.838526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a35b68-a94b-4200-85e0-6437e7fa8923 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:36.838799 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a35b68-a94b-4200-85e0-6437e7fa8923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:36.859259 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d0a35b68-a94b-4200-85e0-6437e7fa8923 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:36.860478 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d0a35b68-a94b-4200-85e0-6437e7fa8923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:36.868199 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1258/2541] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:42:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:36.872310 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd90e08-e79f-41ba-92c7-e0b5452bb26a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:36.881786 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd90e08-e79f-41ba-92c7-e0b5452bb26a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:36.886345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329f05c-19af-49fa-a89f-fec44074b044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:36.886933 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329f05c-19af-49fa-a89f-fec44074b044 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:36.887714 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329f05c-19af-49fa-a89f-fec44074b044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:36.887981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329f05c-19af-49fa-a89f-fec44074b044 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:36.888646 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f329f05c-19af-49fa-a89f-fec44074b044 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:42:36.888944 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f329f05c-19af-49fa-a89f-fec44074b044 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:36.890555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f329f05c-19af-49fa-a89f-fec44074b044 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:36.901937 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1284/2542] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:42:36.906624 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a35b68-a94b-4200-85e0-6437e7fa8923 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:36.915705 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a35b68-a94b-4200-85e0-6437e7fa8923 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:36.920447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e849f-83ae-4a4b-81f3-77dcaddc30fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:36.921035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e849f-83ae-4a4b-81f3-77dcaddc30fa admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:36.921315 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e849f-83ae-4a4b-81f3-77dcaddc30fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:36.921591 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e849f-83ae-4a4b-81f3-77dcaddc30fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:36.922258 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e849f-83ae-4a4b-81f3-77dcaddc30fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:36.924159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e849f-83ae-4a4b-81f3-77dcaddc30fa admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:36.956055 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1259/2543] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:42:36 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:36.960328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329f05c-19af-49fa-a89f-fec44074b044 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:36.970945 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329f05c-19af-49fa-a89f-fec44074b044 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:36.991305 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:37.029904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e170f9-99bf-41f1-832a-43b79f07b307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:37.030536 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e170f9-99bf-41f1-832a-43b79f07b307 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:37.030822 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e170f9-99bf-41f1-832a-43b79f07b307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:37.031100 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e170f9-99bf-41f1-832a-43b79f07b307 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:37.031729 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e170f9-99bf-41f1-832a-43b79f07b307 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:37.033634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e170f9-99bf-41f1-832a-43b79f07b307 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:37.056598 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1285/2544] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:42:36 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:37.060924 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e849f-83ae-4a4b-81f3-77dcaddc30fa admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:37.071606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e849f-83ae-4a4b-81f3-77dcaddc30fa admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:37.076776 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c204b2-6167-437b-999e-9d2f0504d30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:37.077400 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c204b2-6167-437b-999e-9d2f0504d30d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:37.077688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c204b2-6167-437b-999e-9d2f0504d30d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:37.077993 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c204b2-6167-437b-999e-9d2f0504d30d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:37.078682 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c204b2-6167-437b-999e-9d2f0504d30d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:37.080370 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c204b2-6167-437b-999e-9d2f0504d30d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:37.091208 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1260/2545] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:37.095269 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e170f9-99bf-41f1-832a-43b79f07b307 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:37.105252 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e170f9-99bf-41f1-832a-43b79f07b307 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:37.110049 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a03738-360d-4a95-a8ce-1d015d822479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:37.110686 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a03738-360d-4a95-a8ce-1d015d822479 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:37.110971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a03738-360d-4a95-a8ce-1d015d822479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:37.111267 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a03738-360d-4a95-a8ce-1d015d822479 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:37.111964 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a03738-360d-4a95-a8ce-1d015d822479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:37.113774 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a03738-360d-4a95-a8ce-1d015d822479 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:37.125006 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1286/2546] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:37.129128 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c204b2-6167-437b-999e-9d2f0504d30d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:37.139953 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c204b2-6167-437b-999e-9d2f0504d30d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:37.144528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edaad63-6ba6-406d-bd2f-40732b5325d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:37.145168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edaad63-6ba6-406d-bd2f-40732b5325d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:37.145452 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edaad63-6ba6-406d-bd2f-40732b5325d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:37.145729 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edaad63-6ba6-406d-bd2f-40732b5325d3 admin admin] PATH_INFO: `/v3/projects/155c0ff037974845857684d50896c11e/users/da8dfb13cf324220934171c5867c07e4/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:37.160974 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edaad63-6ba6-406d-bd2f-40732b5325d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=155c0ff037974845857684d50896c11e, user_id=da8dfb13cf324220934171c5867c07e4, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:37.163652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edaad63-6ba6-406d-bd2f-40732b5325d3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:37.176322 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1261/2547] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:37 2022] PUT /identity/v3/projects/155c0ff037974845857684d50896c11e/users/da8dfb13cf324220934171c5867c07e4/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:37.186571 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c0eef-37f1-4ac1-99af-58c209681ac8 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:37.187141 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c0eef-37f1-4ac1-99af-58c209681ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:37.187273 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c0eef-37f1-4ac1-99af-58c209681ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:37.214678 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-4f4c0eef-37f1-4ac1-99af-58c209681ac8 None None] MFA Rules not processed for user `da8dfb13cf324220934171c5867c07e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:37.216425 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4f4c0eef-37f1-4ac1-99af-58c209681ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:37.286303 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1287/2548] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:42:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:37.297032 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edaad63-6ba6-406d-bd2f-40732b5325d3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:37.298716 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0edaad63-6ba6-406d-bd2f-40732b5325d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:37.354803 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edaad63-6ba6-406d-bd2f-40732b5325d3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:37.358418 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc6ab4-1e57-4e1b-9c45-b9b236e40cc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:37.359023 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc6ab4-1e57-4e1b-9c45-b9b236e40cc3 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:37.359307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc6ab4-1e57-4e1b-9c45-b9b236e40cc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:37.359586 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc6ab4-1e57-4e1b-9c45-b9b236e40cc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:37.367734 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bc6ab4-1e57-4e1b-9c45-b9b236e40cc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:37.369864 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bc6ab4-1e57-4e1b-9c45-b9b236e40cc3 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:37.379659 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1262/2549] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:41.502967 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c0eef-37f1-4ac1-99af-58c209681ac8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:41.566009 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c0eef-37f1-4ac1-99af-58c209681ac8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:41.570616 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb262a8-5541-4863-a2ce-049d98308caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:41.571267 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb262a8-5541-4863-a2ce-049d98308caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:41.571546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb262a8-5541-4863-a2ce-049d98308caf admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:41.571819 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb262a8-5541-4863-a2ce-049d98308caf admin admin] PATH_INFO: `/v3/users/da8dfb13cf324220934171c5867c07e4` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:41.572937 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb262a8-5541-4863-a2ce-049d98308caf admin admin] RBAC: Authorizing `identity:delete_user(user_id=da8dfb13cf324220934171c5867c07e4)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:41.574898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb262a8-5541-4863-a2ce-049d98308caf admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:41.615096 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:41.626851 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bdb262a8-5541-4863-a2ce-049d98308caf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da8dfb13cf324220934171c5867c07e4'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:41.634034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bdb262a8-5541-4863-a2ce-049d98308caf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da8dfb13cf324220934171c5867c07e4'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:41.634857 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bdb262a8-5541-4863-a2ce-049d98308caf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da8dfb13cf324220934171c5867c07e4'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:41.635504 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bdb262a8-5541-4863-a2ce-049d98308caf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da8dfb13cf324220934171c5867c07e4'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:41.648204 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bdb262a8-5541-4863-a2ce-049d98308caf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da8dfb13cf324220934171c5867c07e4'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:41.663862 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1288/2550] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:42:41 2022] DELETE /identity/v3/users/da8dfb13cf324220934171c5867c07e4 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:41.668363 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb733f6-b72f-47cb-a853-af747484007b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:41.669392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb733f6-b72f-47cb-a853-af747484007b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:41.669514 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb733f6-b72f-47cb-a853-af747484007b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:41.693527 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1eb733f6-b72f-47cb-a853-af747484007b None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:41.695142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1eb733f6-b72f-47cb-a853-af747484007b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:41.739047 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:41.761087 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1263/2551] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:42:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:41.770733 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb262a8-5541-4863-a2ce-049d98308caf admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:41.772134 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bdb262a8-5541-4863-a2ce-049d98308caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:41.829129 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb262a8-5541-4863-a2ce-049d98308caf admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:41.832799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb812c91-43ab-4951-b2bc-d82afddb4147 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:41.833395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb812c91-43ab-4951-b2bc-d82afddb4147 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:41.833676 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb812c91-43ab-4951-b2bc-d82afddb4147 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:41.833947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb812c91-43ab-4951-b2bc-d82afddb4147 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:41.843263 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb812c91-43ab-4951-b2bc-d82afddb4147 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:41.844674 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb812c91-43ab-4951-b2bc-d82afddb4147 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:41.853913 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1289/2552] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:42.128197 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb733f6-b72f-47cb-a853-af747484007b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:42.129652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1eb733f6-b72f-47cb-a853-af747484007b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:42.144694 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb733f6-b72f-47cb-a853-af747484007b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:42.150387 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e03d77-be73-4a07-bbd2-e1f0d808f647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:42.151035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e03d77-be73-4a07-bbd2-e1f0d808f647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:42.151599 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e03d77-be73-4a07-bbd2-e1f0d808f647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:42.151879 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e03d77-be73-4a07-bbd2-e1f0d808f647 admin admin] PATH_INFO: `/v3/projects/155c0ff037974845857684d50896c11e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:42.153100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e03d77-be73-4a07-bbd2-e1f0d808f647 admin admin] RBAC: Authorizing `identity:delete_project(project_id=155c0ff037974845857684d50896c11e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:42.155716 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e03d77-be73-4a07-bbd2-e1f0d808f647 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:42.202720 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-36e03d77-be73-4a07-bbd2-e1f0d808f647 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '155c0ff037974845857684d50896c11e', 'request_id': 'req-36e03d77-be73-4a07-bbd2-e1f0d808f647'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:42.208789 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-36e03d77-be73-4a07-bbd2-e1f0d808f647 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '155c0ff037974845857684d50896c11e', 'request_id': 'req-36e03d77-be73-4a07-bbd2-e1f0d808f647'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:42.216483 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-36e03d77-be73-4a07-bbd2-e1f0d808f647 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '155c0ff037974845857684d50896c11e', 'request_id': 'req-36e03d77-be73-4a07-bbd2-e1f0d808f647'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:42.226892 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-36e03d77-be73-4a07-bbd2-e1f0d808f647 admin admin] The token cache is being invalidate because project 155c0ff037974845857684d50896c11e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:42:42.227188 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-36e03d77-be73-4a07-bbd2-e1f0d808f647 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:42.227827 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-36e03d77-be73-4a07-bbd2-e1f0d808f647 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:42.229469 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1264/2553] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:42:42 2022] DELETE /identity/v3/projects/155c0ff037974845857684d50896c11e => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:42.235037 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b538b7c-5107-4f38-8abb-9cdd8057c73e None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:42.235324 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b538b7c-5107-4f38-8abb-9cdd8057c73e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:42.235604 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b538b7c-5107-4f38-8abb-9cdd8057c73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:42.254886 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-9b538b7c-5107-4f38-8abb-9cdd8057c73e None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:42.256010 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9b538b7c-5107-4f38-8abb-9cdd8057c73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:42.315977 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1290/2554] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:42:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:42.324917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd29d2a-8fd5-456c-a2a7-dc650e8e0c66 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:42.325735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd29d2a-8fd5-456c-a2a7-dc650e8e0c66 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:42.326031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd29d2a-8fd5-456c-a2a7-dc650e8e0c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:42.347839 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-edd29d2a-8fd5-456c-a2a7-dc650e8e0c66 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:42.349674 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-edd29d2a-8fd5-456c-a2a7-dc650e8e0c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:42.361027 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1265/2555] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:42:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:42.365225 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b538b7c-5107-4f38-8abb-9cdd8057c73e None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:42.375743 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b538b7c-5107-4f38-8abb-9cdd8057c73e None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:42.381096 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42165f96-5df7-4f64-80b7-22b70b37aed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:42.381707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42165f96-5df7-4f64-80b7-22b70b37aed0 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:42.382124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42165f96-5df7-4f64-80b7-22b70b37aed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:42.382449 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42165f96-5df7-4f64-80b7-22b70b37aed0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:42.383155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42165f96-5df7-4f64-80b7-22b70b37aed0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:42:42.383489 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42165f96-5df7-4f64-80b7-22b70b37aed0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:42.385229 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42165f96-5df7-4f64-80b7-22b70b37aed0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:42.400405 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1291/2556] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:42:42.405941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd29d2a-8fd5-456c-a2a7-dc650e8e0c66 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:42.415332 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd29d2a-8fd5-456c-a2a7-dc650e8e0c66 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:42.420330 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09874e64-4b95-4ba7-ba76-209db648462d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:42.420933 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09874e64-4b95-4ba7-ba76-209db648462d admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:42.421213 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09874e64-4b95-4ba7-ba76-209db648462d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:42.421487 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09874e64-4b95-4ba7-ba76-209db648462d admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:42.422115 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09874e64-4b95-4ba7-ba76-209db648462d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:42.424070 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09874e64-4b95-4ba7-ba76-209db648462d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:42.453747 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1266/2557] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:42:42 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:42.457881 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42165f96-5df7-4f64-80b7-22b70b37aed0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:42.469134 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42165f96-5df7-4f64-80b7-22b70b37aed0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:42.511003 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:42.530153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb6358e-6d1a-4467-8283-72c732b8589d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:42.530828 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6358e-6d1a-4467-8283-72c732b8589d admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:42.531136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6358e-6d1a-4467-8283-72c732b8589d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:42.531433 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6358e-6d1a-4467-8283-72c732b8589d admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:42.532106 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb6358e-6d1a-4467-8283-72c732b8589d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:42.534137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb6358e-6d1a-4467-8283-72c732b8589d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:42.556793 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1292/2558] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:42:42 2022] POST /identity/v3/users => generated 285 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:42.561520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09874e64-4b95-4ba7-ba76-209db648462d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:42.571628 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09874e64-4b95-4ba7-ba76-209db648462d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:42.576760 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c64918-4786-4dcd-b9bf-260e097dfb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:42.577346 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c64918-4786-4dcd-b9bf-260e097dfb89 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:42.577627 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c64918-4786-4dcd-b9bf-260e097dfb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:42.577899 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c64918-4786-4dcd-b9bf-260e097dfb89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:42.578565 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c64918-4786-4dcd-b9bf-260e097dfb89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:42.580151 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c64918-4786-4dcd-b9bf-260e097dfb89 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:42.589824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1267/2559] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:42.593715 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb6358e-6d1a-4467-8283-72c732b8589d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:42.604345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb6358e-6d1a-4467-8283-72c732b8589d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:42.608966 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c015771-c9f1-4670-a63a-37c0b95fd912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:42.609581 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c015771-c9f1-4670-a63a-37c0b95fd912 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:42.610344 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c015771-c9f1-4670-a63a-37c0b95fd912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:42.610344 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c015771-c9f1-4670-a63a-37c0b95fd912 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:42.610874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c015771-c9f1-4670-a63a-37c0b95fd912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:42.612691 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c015771-c9f1-4670-a63a-37c0b95fd912 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:42.625582 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1293/2560] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:42.630350 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c64918-4786-4dcd-b9bf-260e097dfb89 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:42.641089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c64918-4786-4dcd-b9bf-260e097dfb89 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:42.646185 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d3d3b-6e99-4a05-b9d4-867c57b8b506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:42.646870 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d3d3b-6e99-4a05-b9d4-867c57b8b506 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:42.647154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d3d3b-6e99-4a05-b9d4-867c57b8b506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:42.647431 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d3d3b-6e99-4a05-b9d4-867c57b8b506 admin admin] PATH_INFO: `/v3/projects/aa3686c0f91d40a1b8b0e1af5b9e93f9/users/a7eb999b869048479693b4aaee88410f/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:42.662310 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d3d3b-6e99-4a05-b9d4-867c57b8b506 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa3686c0f91d40a1b8b0e1af5b9e93f9, user_id=a7eb999b869048479693b4aaee88410f, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:42.665010 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d3d3b-6e99-4a05-b9d4-867c57b8b506 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:42.678584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1268/2561] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:42 2022] PUT /identity/v3/projects/aa3686c0f91d40a1b8b0e1af5b9e93f9/users/a7eb999b869048479693b4aaee88410f/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:42.689899 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1fd7d-d536-4fdc-9f50-a8f28486d389 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:42.690235 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1fd7d-d536-4fdc-9f50-a8f28486d389 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:42.690513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1fd7d-d536-4fdc-9f50-a8f28486d389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:42.718978 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-afa1fd7d-d536-4fdc-9f50-a8f28486d389 None None] MFA Rules not processed for user `a7eb999b869048479693b4aaee88410f`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:42.720217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-afa1fd7d-d536-4fdc-9f50-a8f28486d389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:42.788220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1294/2562] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:42:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:42.801075 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d3d3b-6e99-4a05-b9d4-867c57b8b506 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:42.802782 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4f5d3d3b-6e99-4a05-b9d4-867c57b8b506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:42.863153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d3d3b-6e99-4a05-b9d4-867c57b8b506 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:42.866910 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c27905-9b62-421c-a2e4-c6c3b6bcc7b5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2c872129414105a112661cc2b477e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:42.867601 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c27905-9b62-421c-a2e4-c6c3b6bcc7b5 service glance] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:42.867887 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c27905-9b62-421c-a2e4-c6c3b6bcc7b5 service glance] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:42.868185 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c27905-9b62-421c-a2e4-c6c3b6bcc7b5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:42.877744 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c27905-9b62-421c-a2e4-c6c3b6bcc7b5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:42.879314 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c27905-9b62-421c-a2e4-c6c3b6bcc7b5 service glance] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:42.883298 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:42.899470 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1269/2563] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:42 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:50.502766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa1fd7d-d536-4fdc-9f50-a8f28486d389 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:50.504171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-afa1fd7d-d536-4fdc-9f50-a8f28486d389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:50.562564 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa1fd7d-d536-4fdc-9f50-a8f28486d389 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:50.566086 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a8e7bd-4d85-4312-b59f-d13e803135d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:50.566716 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8e7bd-4d85-4312-b59f-d13e803135d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:50.566998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8e7bd-4d85-4312-b59f-d13e803135d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:50.567271 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8e7bd-4d85-4312-b59f-d13e803135d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:50.569019 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-55a8e7bd-4d85-4312-b59f-d13e803135d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:50.632602 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a8e7bd-4d85-4312-b59f-d13e803135d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:50.634056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a8e7bd-4d85-4312-b59f-d13e803135d1 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:50.639508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:50.644491 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1295/2564] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:52.903271 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:53.899750 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c27905-9b62-421c-a2e4-c6c3b6bcc7b5 service glance] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:53.901600 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b2c27905-9b62-421c-a2e4-c6c3b6bcc7b5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:53.917641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c27905-9b62-421c-a2e4-c6c3b6bcc7b5 service glance] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:53.926841 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1953b68-5a1d-4ba9-afa0-280da6fe63b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:53.927548 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1953b68-5a1d-4ba9-afa0-280da6fe63b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:53.927835 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1953b68-5a1d-4ba9-afa0-280da6fe63b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:53.928110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1953b68-5a1d-4ba9-afa0-280da6fe63b5 admin admin] PATH_INFO: `/v3/users/024adeb8a8e14d678ae14174d9555152` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:53.929317 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1953b68-5a1d-4ba9-afa0-280da6fe63b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=024adeb8a8e14d678ae14174d9555152)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:53.931407 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1953b68-5a1d-4ba9-afa0-280da6fe63b5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:53.980129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e1953b68-5a1d-4ba9-afa0-280da6fe63b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '024adeb8a8e14d678ae14174d9555152'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:53.987665 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e1953b68-5a1d-4ba9-afa0-280da6fe63b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '024adeb8a8e14d678ae14174d9555152'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:53.988339 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e1953b68-5a1d-4ba9-afa0-280da6fe63b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '024adeb8a8e14d678ae14174d9555152'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:53.995563 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e1953b68-5a1d-4ba9-afa0-280da6fe63b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '024adeb8a8e14d678ae14174d9555152'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:54.011239 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e1953b68-5a1d-4ba9-afa0-280da6fe63b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '024adeb8a8e14d678ae14174d9555152'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:54.017387 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1270/2565] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:42:53 2022] DELETE /identity/v3/users/024adeb8a8e14d678ae14174d9555152 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:54.196201 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a8e7bd-4d85-4312-b59f-d13e803135d1 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:54.210536 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a8e7bd-4d85-4312-b59f-d13e803135d1 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:54.274047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d36ad3-83aa-4fb5-ac78-c978c5c1dc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:54.274708 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d36ad3-83aa-4fb5-ac78-c978c5c1dc7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:54.274988 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d36ad3-83aa-4fb5-ac78-c978c5c1dc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:54.275260 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d36ad3-83aa-4fb5-ac78-c978c5c1dc7c admin admin] PATH_INFO: `/v3/projects/7d5bb8b858e545b2bcede8b26a2188a3` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:54.276394 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d36ad3-83aa-4fb5-ac78-c978c5c1dc7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d5bb8b858e545b2bcede8b26a2188a3)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:54.278651 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d36ad3-83aa-4fb5-ac78-c978c5c1dc7c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:54.327609 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d5d36ad3-83aa-4fb5-ac78-c978c5c1dc7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d5bb8b858e545b2bcede8b26a2188a3', 'request_id': 'req-d5d36ad3-83aa-4fb5-ac78-c978c5c1dc7c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:54.333753 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d5d36ad3-83aa-4fb5-ac78-c978c5c1dc7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d5bb8b858e545b2bcede8b26a2188a3', 'request_id': 'req-d5d36ad3-83aa-4fb5-ac78-c978c5c1dc7c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:54.341709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d5d36ad3-83aa-4fb5-ac78-c978c5c1dc7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d5bb8b858e545b2bcede8b26a2188a3', 'request_id': 'req-d5d36ad3-83aa-4fb5-ac78-c978c5c1dc7c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:54.355412 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d5d36ad3-83aa-4fb5-ac78-c978c5c1dc7c admin admin] The token cache is being invalidate because project 7d5bb8b858e545b2bcede8b26a2188a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:42:54.355696 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d5d36ad3-83aa-4fb5-ac78-c978c5c1dc7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:54.356310 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d5d36ad3-83aa-4fb5-ac78-c978c5c1dc7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:54.357855 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1296/2566] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:42:54 2022] DELETE /identity/v3/projects/7d5bb8b858e545b2bcede8b26a2188a3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:54.362997 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef1ac9-eabd-49c3-8cf1-f2c09814128d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:54.363285 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef1ac9-eabd-49c3-8cf1-f2c09814128d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:54.363607 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef1ac9-eabd-49c3-8cf1-f2c09814128d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:54.383495 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9aef1ac9-eabd-49c3-8cf1-f2c09814128d None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:54.384921 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9aef1ac9-eabd-49c3-8cf1-f2c09814128d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:54.454287 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1271/2567] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:42:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:54.462221 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2cdac-e2f1-4c2e-8b3b-376b82db45b5 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:54.462529 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2cdac-e2f1-4c2e-8b3b-376b82db45b5 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:54.462808 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2cdac-e2f1-4c2e-8b3b-376b82db45b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:54.485224 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d5d2cdac-e2f1-4c2e-8b3b-376b82db45b5 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:54.486468 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d5d2cdac-e2f1-4c2e-8b3b-376b82db45b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:54.494653 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1297/2568] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:42:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:42:54.499592 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef1ac9-eabd-49c3-8cf1-f2c09814128d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:54.509972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef1ac9-eabd-49c3-8cf1-f2c09814128d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:54.515300 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54c757-1cfa-4af5-a4f4-43982d444e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:54.515916 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54c757-1cfa-4af5-a4f4-43982d444e9a admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:54.516192 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54c757-1cfa-4af5-a4f4-43982d444e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:54.516610 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54c757-1cfa-4af5-a4f4-43982d444e9a admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:54.517268 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d54c757-1cfa-4af5-a4f4-43982d444e9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:42:54.517663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d54c757-1cfa-4af5-a4f4-43982d444e9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:54.519315 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d54c757-1cfa-4af5-a4f4-43982d444e9a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:54.533014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1272/2569] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:42:54.537493 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2cdac-e2f1-4c2e-8b3b-376b82db45b5 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:54.548075 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2cdac-e2f1-4c2e-8b3b-376b82db45b5 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:54.552657 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56cea2a-276d-4bf1-b8c7-6394f1515134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:54.553252 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56cea2a-276d-4bf1-b8c7-6394f1515134 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:54.553528 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56cea2a-276d-4bf1-b8c7-6394f1515134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:54.553798 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56cea2a-276d-4bf1-b8c7-6394f1515134 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:54.554472 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56cea2a-276d-4bf1-b8c7-6394f1515134 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:54.556353 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56cea2a-276d-4bf1-b8c7-6394f1515134 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:54.587033 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1298/2570] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:42:54 2022] POST /identity/v3/projects => generated 409 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:54.591080 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54c757-1cfa-4af5-a4f4-43982d444e9a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:54.601819 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54c757-1cfa-4af5-a4f4-43982d444e9a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:54.659719 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a34b5-7e59-4500-bff9-535ac0bfb54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:54.660319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a34b5-7e59-4500-bff9-535ac0bfb54a admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:54.660598 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a34b5-7e59-4500-bff9-535ac0bfb54a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:54.660870 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a34b5-7e59-4500-bff9-535ac0bfb54a admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:54.661511 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a34b5-7e59-4500-bff9-535ac0bfb54a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:54.663450 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a34b5-7e59-4500-bff9-535ac0bfb54a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:54.671340 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:54.691145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1273/2571] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:42:54 2022] POST /identity/v3/users => generated 305 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:54.692297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56cea2a-276d-4bf1-b8c7-6394f1515134 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:54.703440 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56cea2a-276d-4bf1-b8c7-6394f1515134 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:54.708355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35306851-1eba-40fd-a5c4-72762f5b52cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:54.708960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35306851-1eba-40fd-a5c4-72762f5b52cf admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:54.709231 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35306851-1eba-40fd-a5c4-72762f5b52cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:54.709615 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35306851-1eba-40fd-a5c4-72762f5b52cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:54.710293 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35306851-1eba-40fd-a5c4-72762f5b52cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:54.711872 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35306851-1eba-40fd-a5c4-72762f5b52cf admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:54.723065 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1299/2572] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:42:54.727224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a34b5-7e59-4500-bff9-535ac0bfb54a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:54.737742 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a34b5-7e59-4500-bff9-535ac0bfb54a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:54.742356 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e6374-966a-438f-b4ea-27bbf1ff249d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:54.742954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e6374-966a-438f-b4ea-27bbf1ff249d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:54.743238 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e6374-966a-438f-b4ea-27bbf1ff249d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:54.743513 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e6374-966a-438f-b4ea-27bbf1ff249d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:54.744159 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e6374-966a-438f-b4ea-27bbf1ff249d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:54.745772 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e6374-966a-438f-b4ea-27bbf1ff249d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:54.756644 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1274/2573] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:54.761022 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35306851-1eba-40fd-a5c4-72762f5b52cf admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:54.770995 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35306851-1eba-40fd-a5c4-72762f5b52cf admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:54.775591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646731df-1619-4749-b3e9-e62baa3f916a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:54.776240 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646731df-1619-4749-b3e9-e62baa3f916a admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:54.776522 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646731df-1619-4749-b3e9-e62baa3f916a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:54.776801 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646731df-1619-4749-b3e9-e62baa3f916a admin admin] PATH_INFO: `/v3/projects/9911670951bd4c13adc484b562f7683e/users/4fa10fe036be4bfdb75c93efeb7ac934/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:54.792920 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646731df-1619-4749-b3e9-e62baa3f916a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9911670951bd4c13adc484b562f7683e, user_id=4fa10fe036be4bfdb75c93efeb7ac934, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:54.797442 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646731df-1619-4749-b3e9-e62baa3f916a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:54.814688 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1300/2574] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:54 2022] PUT /identity/v3/projects/9911670951bd4c13adc484b562f7683e/users/4fa10fe036be4bfdb75c93efeb7ac934/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:54.825281 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f7144-bc55-4632-b172-499058ba8f77 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:54.825571 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f7144-bc55-4632-b172-499058ba8f77 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:54.825846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f7144-bc55-4632-b172-499058ba8f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:54.854697 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-646f7144-bc55-4632-b172-499058ba8f77 None None] MFA Rules not processed for user `4fa10fe036be4bfdb75c93efeb7ac934`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:54.856016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-646f7144-bc55-4632-b172-499058ba8f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:54.923355 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1275/2575] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:42:54 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:54.928090 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646731df-1619-4749-b3e9-e62baa3f916a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:54.963077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:54.999566 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646731df-1619-4749-b3e9-e62baa3f916a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:55.004159 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba76c4a-011a-4b34-8e02-79b8cce5a786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:55.004759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba76c4a-011a-4b34-8e02-79b8cce5a786 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:55.005036 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba76c4a-011a-4b34-8e02-79b8cce5a786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:55.005308 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba76c4a-011a-4b34-8e02-79b8cce5a786 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:55.006120 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba76c4a-011a-4b34-8e02-79b8cce5a786 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:55.008051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba76c4a-011a-4b34-8e02-79b8cce5a786 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:55.036118 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1301/2576] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:42:54 2022] POST /identity/v3/projects => generated 407 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:55.040212 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f7144-bc55-4632-b172-499058ba8f77 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:55.052016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f7144-bc55-4632-b172-499058ba8f77 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:55.109954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8e618-320b-4775-b1b1-6ae823e1566a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:55.110556 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8e618-320b-4775-b1b1-6ae823e1566a admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:55.110837 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8e618-320b-4775-b1b1-6ae823e1566a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:55.111110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8e618-320b-4775-b1b1-6ae823e1566a admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:55.111745 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8e618-320b-4775-b1b1-6ae823e1566a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:55.114091 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8e618-320b-4775-b1b1-6ae823e1566a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:55.136402 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1276/2577] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:42:55 2022] POST /identity/v3/users => generated 310 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:55.140988 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba76c4a-011a-4b34-8e02-79b8cce5a786 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:55.151389 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba76c4a-011a-4b34-8e02-79b8cce5a786 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:55.156891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a88b519-9026-437b-ab3a-f91db5fa780e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:55.157555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88b519-9026-437b-ab3a-f91db5fa780e admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:55.158388 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88b519-9026-437b-ab3a-f91db5fa780e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:55.158665 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88b519-9026-437b-ab3a-f91db5fa780e admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:55.159323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a88b519-9026-437b-ab3a-f91db5fa780e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:55.160991 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a88b519-9026-437b-ab3a-f91db5fa780e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:55.175223 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1302/2578] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:55.180923 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8e618-320b-4775-b1b1-6ae823e1566a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:55.195269 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8e618-320b-4775-b1b1-6ae823e1566a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:55.201949 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee994125-bc4b-4a69-84e2-9c32261cef0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:55.203281 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee994125-bc4b-4a69-84e2-9c32261cef0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:55.203569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee994125-bc4b-4a69-84e2-9c32261cef0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:55.203852 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee994125-bc4b-4a69-84e2-9c32261cef0d admin admin] PATH_INFO: `/v3/projects/c826282f8d804f968f7811b0dc4a6fa5/users/4adaa2ed068c4f0083bc1e9376e931fb/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:55.220415 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee994125-bc4b-4a69-84e2-9c32261cef0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c826282f8d804f968f7811b0dc4a6fa5, user_id=4adaa2ed068c4f0083bc1e9376e931fb, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:55.223715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee994125-bc4b-4a69-84e2-9c32261cef0d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:55.238140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1277/2579] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:55 2022] PUT /identity/v3/projects/c826282f8d804f968f7811b0dc4a6fa5/users/4adaa2ed068c4f0083bc1e9376e931fb/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:55.242366 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a88b519-9026-437b-ab3a-f91db5fa780e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:55.253055 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a88b519-9026-437b-ab3a-f91db5fa780e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:55.310123 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07082d-291e-429b-9893-8a3890a0e906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:55.311325 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07082d-291e-429b-9893-8a3890a0e906 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:55.311607 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07082d-291e-429b-9893-8a3890a0e906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:55.311882 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07082d-291e-429b-9893-8a3890a0e906 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:55.312528 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc07082d-291e-429b-9893-8a3890a0e906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:55.314122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc07082d-291e-429b-9893-8a3890a0e906 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:55.323855 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1303/2580] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:55.328972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee994125-bc4b-4a69-84e2-9c32261cef0d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:55.340017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee994125-bc4b-4a69-84e2-9c32261cef0d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:55.344761 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7451681-3830-46d0-a5c4-003fcb63680b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:55.345417 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7451681-3830-46d0-a5c4-003fcb63680b admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:55.345696 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7451681-3830-46d0-a5c4-003fcb63680b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:55.345968 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7451681-3830-46d0-a5c4-003fcb63680b admin admin] PATH_INFO: `/v3/projects/c826282f8d804f968f7811b0dc4a6fa5/users/4adaa2ed068c4f0083bc1e9376e931fb/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:55.347880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7451681-3830-46d0-a5c4-003fcb63680b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c826282f8d804f968f7811b0dc4a6fa5, user_id=4adaa2ed068c4f0083bc1e9376e931fb, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:55.350525 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7451681-3830-46d0-a5c4-003fcb63680b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:55.360573 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1278/2581] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:55 2022] PUT /identity/v3/projects/c826282f8d804f968f7811b0dc4a6fa5/users/4adaa2ed068c4f0083bc1e9376e931fb/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:42:55.372531 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d24cd-6e78-4ea7-a358-6b97118dbb6a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:55.372822 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d24cd-6e78-4ea7-a358-6b97118dbb6a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:55.373735 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d24cd-6e78-4ea7-a358-6b97118dbb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:55.402094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f79d24cd-6e78-4ea7-a358-6b97118dbb6a None None] MFA Rules not processed for user `4adaa2ed068c4f0083bc1e9376e931fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:55.403602 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f79d24cd-6e78-4ea7-a358-6b97118dbb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:55.471176 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1304/2582] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:42:55 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:55.482043 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7451681-3830-46d0-a5c4-003fcb63680b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:55.483800 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e7451681-3830-46d0-a5c4-003fcb63680b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:55.515070 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:55.551062 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7451681-3830-46d0-a5c4-003fcb63680b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:55.554696 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1ecb3-a9e6-403c-ad67-3978b9445c33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:55.555296 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1ecb3-a9e6-403c-ad67-3978b9445c33 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:55.555576 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1ecb3-a9e6-403c-ad67-3978b9445c33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:55.555848 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1ecb3-a9e6-403c-ad67-3978b9445c33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:55.565739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1ecb3-a9e6-403c-ad67-3978b9445c33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:55.567104 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1ecb3-a9e6-403c-ad67-3978b9445c33 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:55.577702 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1279/2583] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:55 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:55.726797 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d24cd-6e78-4ea7-a358-6b97118dbb6a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:55.737944 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d24cd-6e78-4ea7-a358-6b97118dbb6a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:55.742254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a8240-a4c8-4f2b-9100-54d94d53a071 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:55.742908 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a8240-a4c8-4f2b-9100-54d94d53a071 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:55.743197 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a8240-a4c8-4f2b-9100-54d94d53a071 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:55.743532 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a8240-a4c8-4f2b-9100-54d94d53a071 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:55.763661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:55.802258 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a8240-a4c8-4f2b-9100-54d94d53a071 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:55.803702 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a8240-a4c8-4f2b-9100-54d94d53a071 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:55.813542 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1305/2584] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:55 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:05.572219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=17 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:05.808900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=18 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:12.405373 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1ecb3-a9e6-403c-ad67-3978b9445c33 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:12.467079 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1ecb3-a9e6-403c-ad67-3978b9445c33 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:12.472556 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f50ff-1377-4831-9b94-89da2c9e5d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:12.473209 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f50ff-1377-4831-9b94-89da2c9e5d35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:12.473538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f50ff-1377-4831-9b94-89da2c9e5d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:12.473854 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f50ff-1377-4831-9b94-89da2c9e5d35 admin admin] PATH_INFO: `/v3/users/4fa10fe036be4bfdb75c93efeb7ac934` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:12.475253 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f50ff-1377-4831-9b94-89da2c9e5d35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fa10fe036be4bfdb75c93efeb7ac934)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:12.477498 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f50ff-1377-4831-9b94-89da2c9e5d35 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:12.522112 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d41f50ff-1377-4831-9b94-89da2c9e5d35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fa10fe036be4bfdb75c93efeb7ac934'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:12.529232 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d41f50ff-1377-4831-9b94-89da2c9e5d35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fa10fe036be4bfdb75c93efeb7ac934'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:12.529917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d41f50ff-1377-4831-9b94-89da2c9e5d35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fa10fe036be4bfdb75c93efeb7ac934'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:12.537559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d41f50ff-1377-4831-9b94-89da2c9e5d35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fa10fe036be4bfdb75c93efeb7ac934'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:12.549549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d41f50ff-1377-4831-9b94-89da2c9e5d35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fa10fe036be4bfdb75c93efeb7ac934'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:12.555438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1280/2585] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:12 2022] DELETE /identity/v3/users/4fa10fe036be4bfdb75c93efeb7ac934 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:12.559865 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772e9f1-7f10-4036-9b20-9fea7ac6f139 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:12.560360 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772e9f1-7f10-4036-9b20-9fea7ac6f139 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:12.560469 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772e9f1-7f10-4036-9b20-9fea7ac6f139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:12.580118 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-8772e9f1-7f10-4036-9b20-9fea7ac6f139 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:12.581300 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8772e9f1-7f10-4036-9b20-9fea7ac6f139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:12.641090 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1306/2586] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:12.651135 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f50ff-1377-4831-9b94-89da2c9e5d35 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:12.652608 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d41f50ff-1377-4831-9b94-89da2c9e5d35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:12.722157 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f50ff-1377-4831-9b94-89da2c9e5d35 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:12.725823 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0ead2-99f8-4e84-a77b-5ae6a8e79588 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:12.726460 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0ead2-99f8-4e84-a77b-5ae6a8e79588 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:12.726745 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0ead2-99f8-4e84-a77b-5ae6a8e79588 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:12.727016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0ead2-99f8-4e84-a77b-5ae6a8e79588 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:12.736445 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0ead2-99f8-4e84-a77b-5ae6a8e79588 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:12.737876 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0ead2-99f8-4e84-a77b-5ae6a8e79588 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:12.747129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1281/2587] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:13.020259 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772e9f1-7f10-4036-9b20-9fea7ac6f139 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:13.021685 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8772e9f1-7f10-4036-9b20-9fea7ac6f139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:13.036234 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772e9f1-7f10-4036-9b20-9fea7ac6f139 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:13.041398 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a99c49-b90e-4102-b567-4afe5d2c871a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:13.042032 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a99c49-b90e-4102-b567-4afe5d2c871a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:13.042351 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a99c49-b90e-4102-b567-4afe5d2c871a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:13.042627 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a99c49-b90e-4102-b567-4afe5d2c871a admin admin] PATH_INFO: `/v3/projects/9911670951bd4c13adc484b562f7683e` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:13.051217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a99c49-b90e-4102-b567-4afe5d2c871a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9911670951bd4c13adc484b562f7683e)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:13.053181 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a99c49-b90e-4102-b567-4afe5d2c871a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:13.099474 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-80a99c49-b90e-4102-b567-4afe5d2c871a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9911670951bd4c13adc484b562f7683e', 'request_id': 'req-80a99c49-b90e-4102-b567-4afe5d2c871a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.105531 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-80a99c49-b90e-4102-b567-4afe5d2c871a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9911670951bd4c13adc484b562f7683e', 'request_id': 'req-80a99c49-b90e-4102-b567-4afe5d2c871a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.111688 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-80a99c49-b90e-4102-b567-4afe5d2c871a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9911670951bd4c13adc484b562f7683e', 'request_id': 'req-80a99c49-b90e-4102-b567-4afe5d2c871a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.123987 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-80a99c49-b90e-4102-b567-4afe5d2c871a admin admin] The token cache is being invalidate because project 9911670951bd4c13adc484b562f7683e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:13.124301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-80a99c49-b90e-4102-b567-4afe5d2c871a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.125561 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-80a99c49-b90e-4102-b567-4afe5d2c871a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.127172 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1307/2588] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:13 2022] DELETE /identity/v3/projects/9911670951bd4c13adc484b562f7683e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:13.131198 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0ead2-99f8-4e84-a77b-5ae6a8e79588 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:13.140771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0ead2-99f8-4e84-a77b-5ae6a8e79588 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:13.159076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:13.198475 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac76aff-3ef4-4a4b-8125-7763a4e70d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:13.199100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac76aff-3ef4-4a4b-8125-7763a4e70d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:13.199409 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac76aff-3ef4-4a4b-8125-7763a4e70d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:13.199685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac76aff-3ef4-4a4b-8125-7763a4e70d40 admin admin] PATH_INFO: `/v3/users/4adaa2ed068c4f0083bc1e9376e931fb` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:13.200794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac76aff-3ef4-4a4b-8125-7763a4e70d40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4adaa2ed068c4f0083bc1e9376e931fb)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:13.203287 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac76aff-3ef4-4a4b-8125-7763a4e70d40 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:13.252349 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9ac76aff-3ef4-4a4b-8125-7763a4e70d40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4adaa2ed068c4f0083bc1e9376e931fb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.259197 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9ac76aff-3ef4-4a4b-8125-7763a4e70d40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4adaa2ed068c4f0083bc1e9376e931fb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.259901 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9ac76aff-3ef4-4a4b-8125-7763a4e70d40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4adaa2ed068c4f0083bc1e9376e931fb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.267053 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9ac76aff-3ef4-4a4b-8125-7763a4e70d40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4adaa2ed068c4f0083bc1e9376e931fb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.279207 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9ac76aff-3ef4-4a4b-8125-7763a4e70d40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4adaa2ed068c4f0083bc1e9376e931fb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.285385 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1282/2589] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:13 2022] DELETE /identity/v3/users/4adaa2ed068c4f0083bc1e9376e931fb => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:13.565730 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a99c49-b90e-4102-b567-4afe5d2c871a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:13.567307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-80a99c49-b90e-4102-b567-4afe5d2c871a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:13.580758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a99c49-b90e-4102-b567-4afe5d2c871a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:13.639782 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010908ae-dde1-4a7e-8fe9-bb7c8dabf1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:13.640442 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010908ae-dde1-4a7e-8fe9-bb7c8dabf1a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:13.640717 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010908ae-dde1-4a7e-8fe9-bb7c8dabf1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:13.641012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010908ae-dde1-4a7e-8fe9-bb7c8dabf1a2 admin admin] PATH_INFO: `/v3/projects/c826282f8d804f968f7811b0dc4a6fa5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:13.642117 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010908ae-dde1-4a7e-8fe9-bb7c8dabf1a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c826282f8d804f968f7811b0dc4a6fa5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:13.644039 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010908ae-dde1-4a7e-8fe9-bb7c8dabf1a2 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:13.674906 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:13.693049 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-010908ae-dde1-4a7e-8fe9-bb7c8dabf1a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c826282f8d804f968f7811b0dc4a6fa5', 'request_id': 'req-010908ae-dde1-4a7e-8fe9-bb7c8dabf1a2'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.699186 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-010908ae-dde1-4a7e-8fe9-bb7c8dabf1a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c826282f8d804f968f7811b0dc4a6fa5', 'request_id': 'req-010908ae-dde1-4a7e-8fe9-bb7c8dabf1a2'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.704921 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-010908ae-dde1-4a7e-8fe9-bb7c8dabf1a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c826282f8d804f968f7811b0dc4a6fa5', 'request_id': 'req-010908ae-dde1-4a7e-8fe9-bb7c8dabf1a2'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.716903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-010908ae-dde1-4a7e-8fe9-bb7c8dabf1a2 admin admin] The token cache is being invalidate because project c826282f8d804f968f7811b0dc4a6fa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:13.717184 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-010908ae-dde1-4a7e-8fe9-bb7c8dabf1a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.717793 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-010908ae-dde1-4a7e-8fe9-bb7c8dabf1a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.719643 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1308/2590] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:13 2022] DELETE /identity/v3/projects/c826282f8d804f968f7811b0dc4a6fa5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:13.724303 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57f2aa-0b63-4f99-a94d-d945d0455574 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:13.724763 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57f2aa-0b63-4f99-a94d-d945d0455574 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:13.724921 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57f2aa-0b63-4f99-a94d-d945d0455574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:13.745354 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fb57f2aa-0b63-4f99-a94d-d945d0455574 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:13.746724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fb57f2aa-0b63-4f99-a94d-d945d0455574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:13.805886 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1283/2591] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:43:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:13.814241 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7ff04-e5a9-40df-b69d-eb6a3eaba1ee None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:13.814532 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7ff04-e5a9-40df-b69d-eb6a3eaba1ee None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:13.814807 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7ff04-e5a9-40df-b69d-eb6a3eaba1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:13.834340 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-83f7ff04-e5a9-40df-b69d-eb6a3eaba1ee None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:13.835557 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-83f7ff04-e5a9-40df-b69d-eb6a3eaba1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:13.843579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1309/2592] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:43:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:13.847736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57f2aa-0b63-4f99-a94d-d945d0455574 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:13.857505 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57f2aa-0b63-4f99-a94d-d945d0455574 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:13.862653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e93567-9e5e-456f-89ec-facf4d3fd95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:13.863252 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e93567-9e5e-456f-89ec-facf4d3fd95a admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:13.863540 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e93567-9e5e-456f-89ec-facf4d3fd95a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:13.863820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e93567-9e5e-456f-89ec-facf4d3fd95a admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:13.864470 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e93567-9e5e-456f-89ec-facf4d3fd95a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:43:13.864778 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e93567-9e5e-456f-89ec-facf4d3fd95a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:13.866408 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e93567-9e5e-456f-89ec-facf4d3fd95a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:13.878617 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1284/2593] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:43:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:43:13.882796 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7ff04-e5a9-40df-b69d-eb6a3eaba1ee None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:13.892003 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7ff04-e5a9-40df-b69d-eb6a3eaba1ee None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:13.896910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c85292-d217-4502-b388-a18618f6c3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:13.897511 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c85292-d217-4502-b388-a18618f6c3a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:13.897790 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c85292-d217-4502-b388-a18618f6c3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:13.898066 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c85292-d217-4502-b388-a18618f6c3a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:13.898769 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c85292-d217-4502-b388-a18618f6c3a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:13.900645 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c85292-d217-4502-b388-a18618f6c3a2 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:13.930685 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1310/2594] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:13 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:13.934861 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e93567-9e5e-456f-89ec-facf4d3fd95a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:13.945635 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e93567-9e5e-456f-89ec-facf4d3fd95a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:13.972565 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:14.007205 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d81ffb-c8dd-4b6c-a047-252fdb5a5a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:14.007815 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d81ffb-c8dd-4b6c-a047-252fdb5a5a9a admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:14.008205 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d81ffb-c8dd-4b6c-a047-252fdb5a5a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:14.008508 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d81ffb-c8dd-4b6c-a047-252fdb5a5a9a admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:14.009118 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d81ffb-c8dd-4b6c-a047-252fdb5a5a9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:14.011089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d81ffb-c8dd-4b6c-a047-252fdb5a5a9a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:14.032765 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1285/2595] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:13 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:14.036877 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c85292-d217-4502-b388-a18618f6c3a2 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:14.048724 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c85292-d217-4502-b388-a18618f6c3a2 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:14.053737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e7d05-e697-406a-8f74-b7b50e0be5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:14.054374 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e7d05-e697-406a-8f74-b7b50e0be5b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:14.054658 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e7d05-e697-406a-8f74-b7b50e0be5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:14.054933 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e7d05-e697-406a-8f74-b7b50e0be5b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:14.055612 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53e7d05-e697-406a-8f74-b7b50e0be5b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:14.057239 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53e7d05-e697-406a-8f74-b7b50e0be5b4 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:14.067481 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1311/2596] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:14.071483 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d81ffb-c8dd-4b6c-a047-252fdb5a5a9a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:14.081458 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d81ffb-c8dd-4b6c-a047-252fdb5a5a9a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:14.086032 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa41aa-77a9-430e-9f07-829b545b5ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:14.086763 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa41aa-77a9-430e-9f07-829b545b5ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:14.087103 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa41aa-77a9-430e-9f07-829b545b5ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:14.087333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa41aa-77a9-430e-9f07-829b545b5ab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:14.088061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aa41aa-77a9-430e-9f07-829b545b5ab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:14.089732 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aa41aa-77a9-430e-9f07-829b545b5ab2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:14.104091 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1286/2597] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:43:14.105472 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e7d05-e697-406a-8f74-b7b50e0be5b4 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:14.117797 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e7d05-e697-406a-8f74-b7b50e0be5b4 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:14.122709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a225052-4d72-48a1-bd26-708878794787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:14.123362 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a225052-4d72-48a1-bd26-708878794787 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:14.123765 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a225052-4d72-48a1-bd26-708878794787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:14.124042 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a225052-4d72-48a1-bd26-708878794787 admin admin] PATH_INFO: `/v3/projects/2d26b3d2008e43e3a277dbf3ca09d34a/users/bdfff719bbbe4a1fbfcc6e7f617f62e1/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:14.138952 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a225052-4d72-48a1-bd26-708878794787 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d26b3d2008e43e3a277dbf3ca09d34a, user_id=bdfff719bbbe4a1fbfcc6e7f617f62e1, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:14.141702 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a225052-4d72-48a1-bd26-708878794787 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:14.154655 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1312/2598] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:14 2022] PUT /identity/v3/projects/2d26b3d2008e43e3a277dbf3ca09d34a/users/bdfff719bbbe4a1fbfcc6e7f617f62e1/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:14.166909 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a9693-6065-4359-ab01-895957318fdf None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:14.167200 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a9693-6065-4359-ab01-895957318fdf None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:14.167501 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a9693-6065-4359-ab01-895957318fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:14.195880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-083a9693-6065-4359-ab01-895957318fdf None None] MFA Rules not processed for user `bdfff719bbbe4a1fbfcc6e7f617f62e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:14.197024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-083a9693-6065-4359-ab01-895957318fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:14.261676 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1287/2599] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:43:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:14.266277 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a225052-4d72-48a1-bd26-708878794787 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:14.327070 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a225052-4d72-48a1-bd26-708878794787 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:14.331644 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f6fb3-e3b4-4e5c-9fcf-de088929f13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:14.332234 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f6fb3-e3b4-4e5c-9fcf-de088929f13e admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:14.332516 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f6fb3-e3b4-4e5c-9fcf-de088929f13e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:14.332785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f6fb3-e3b4-4e5c-9fcf-de088929f13e admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:14.333404 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f6fb3-e3b4-4e5c-9fcf-de088929f13e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:14.335325 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f6fb3-e3b4-4e5c-9fcf-de088929f13e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:14.366140 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1313/2600] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:14 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:43:14.370788 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a9693-6065-4359-ab01-895957318fdf None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:14.381546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a9693-6065-4359-ab01-895957318fdf None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:14.438271 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d34bee-9380-4d02-a6fe-2c1e822db02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:14.438876 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d34bee-9380-4d02-a6fe-2c1e822db02b admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:14.439158 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d34bee-9380-4d02-a6fe-2c1e822db02b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:14.439433 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d34bee-9380-4d02-a6fe-2c1e822db02b admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:14.440067 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d34bee-9380-4d02-a6fe-2c1e822db02b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:14.441972 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d34bee-9380-4d02-a6fe-2c1e822db02b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:14.462271 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1288/2601] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:14 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:14.466285 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f6fb3-e3b4-4e5c-9fcf-de088929f13e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:14.477282 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f6fb3-e3b4-4e5c-9fcf-de088929f13e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:14.481907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31b9f01-2e52-4638-b650-9e6fbb5d8e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:14.482539 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b9f01-2e52-4638-b650-9e6fbb5d8e93 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:14.482824 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b9f01-2e52-4638-b650-9e6fbb5d8e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:14.483097 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b9f01-2e52-4638-b650-9e6fbb5d8e93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:14.483737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31b9f01-2e52-4638-b650-9e6fbb5d8e93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:14.485455 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31b9f01-2e52-4638-b650-9e6fbb5d8e93 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:14.495893 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1314/2602] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:14.499821 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d34bee-9380-4d02-a6fe-2c1e822db02b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:14.509650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d34bee-9380-4d02-a6fe-2c1e822db02b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:14.514248 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6a0d6-1f3c-49c6-8f75-2a43e759b01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:14.514887 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6a0d6-1f3c-49c6-8f75-2a43e759b01e admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:14.515171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6a0d6-1f3c-49c6-8f75-2a43e759b01e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:14.515967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6a0d6-1f3c-49c6-8f75-2a43e759b01e admin admin] PATH_INFO: `/v3/projects/40fd89822af7467aa7d33b7fc96423fb/users/dac4549def66408f8d12e102a83a33df/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:14.530756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b6a0d6-1f3c-49c6-8f75-2a43e759b01e admin admin] RBAC: Authorizing `identity:create_grant(project_id=40fd89822af7467aa7d33b7fc96423fb, user_id=dac4549def66408f8d12e102a83a33df, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:14.534066 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b6a0d6-1f3c-49c6-8f75-2a43e759b01e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:14.547215 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1289/2603] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:14 2022] PUT /identity/v3/projects/40fd89822af7467aa7d33b7fc96423fb/users/dac4549def66408f8d12e102a83a33df/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:14.551124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31b9f01-2e52-4638-b650-9e6fbb5d8e93 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:14.560942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31b9f01-2e52-4638-b650-9e6fbb5d8e93 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:14.581071 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:14.618813 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702dec4-85b5-40c4-8b53-e3b0451c31e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:14.619419 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702dec4-85b5-40c4-8b53-e3b0451c31e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:14.619695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702dec4-85b5-40c4-8b53-e3b0451c31e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:14.619967 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702dec4-85b5-40c4-8b53-e3b0451c31e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:14.620607 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702dec4-85b5-40c4-8b53-e3b0451c31e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:14.622532 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702dec4-85b5-40c4-8b53-e3b0451c31e8 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:14.632900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1315/2604] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:14.637181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6a0d6-1f3c-49c6-8f75-2a43e759b01e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:14.643306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:14.649561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6a0d6-1f3c-49c6-8f75-2a43e759b01e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:14.654353 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e5adc-cd7e-4eac-b679-13207b8f29c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:14.654998 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e5adc-cd7e-4eac-b679-13207b8f29c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:14.655543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e5adc-cd7e-4eac-b679-13207b8f29c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:14.655814 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e5adc-cd7e-4eac-b679-13207b8f29c1 admin admin] PATH_INFO: `/v3/projects/40fd89822af7467aa7d33b7fc96423fb/users/dac4549def66408f8d12e102a83a33df/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:14.658340 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135e5adc-cd7e-4eac-b679-13207b8f29c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40fd89822af7467aa7d33b7fc96423fb, user_id=dac4549def66408f8d12e102a83a33df, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:14.660944 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135e5adc-cd7e-4eac-b679-13207b8f29c1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:14.670906 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1290/2605] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:14 2022] PUT /identity/v3/projects/40fd89822af7467aa7d33b7fc96423fb/users/dac4549def66408f8d12e102a83a33df/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:14.681597 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2751bf-c55c-4b13-a443-fc8290d4a357 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:14.681881 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2751bf-c55c-4b13-a443-fc8290d4a357 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:14.682152 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2751bf-c55c-4b13-a443-fc8290d4a357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:14.710432 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2b2751bf-c55c-4b13-a443-fc8290d4a357 None None] MFA Rules not processed for user `dac4549def66408f8d12e102a83a33df`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:14.711643 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2b2751bf-c55c-4b13-a443-fc8290d4a357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:14.778842 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1316/2606] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:43:14 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:14.791614 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e5adc-cd7e-4eac-b679-13207b8f29c1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:14.793397 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-135e5adc-cd7e-4eac-b679-13207b8f29c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:14.851392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e5adc-cd7e-4eac-b679-13207b8f29c1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:14.855132 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81da18-7857-4c82-8515-f4a3bf2a5f9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:14.855756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81da18-7857-4c82-8515-f4a3bf2a5f9e service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:14.856041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81da18-7857-4c82-8515-f4a3bf2a5f9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:14.856320 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81da18-7857-4c82-8515-f4a3bf2a5f9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:14.865805 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a81da18-7857-4c82-8515-f4a3bf2a5f9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:14.867267 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a81da18-7857-4c82-8515-f4a3bf2a5f9e service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:14.876584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1291/2607] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:14 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:14.947287 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2751bf-c55c-4b13-a443-fc8290d4a357 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:14.948754 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2b2751bf-c55c-4b13-a443-fc8290d4a357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:14.960728 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2751bf-c55c-4b13-a443-fc8290d4a357 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:14.964623 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6296c4-c683-48b4-b74d-5ef3c3823f1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:14.965300 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6296c4-c683-48b4-b74d-5ef3c3823f1a service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:14.965652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6296c4-c683-48b4-b74d-5ef3c3823f1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:14.966002 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6296c4-c683-48b4-b74d-5ef3c3823f1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:15.024149 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6296c4-c683-48b4-b74d-5ef3c3823f1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:15.025569 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6296c4-c683-48b4-b74d-5ef3c3823f1a service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:15.035055 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1317/2608] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:15.322416 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81da18-7857-4c82-8515-f4a3bf2a5f9e service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:15.391133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81da18-7857-4c82-8515-f4a3bf2a5f9e service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:15.396496 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ea96f-22eb-4c3c-894e-8726a3a9335a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:15.397132 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ea96f-22eb-4c3c-894e-8726a3a9335a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:15.397417 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ea96f-22eb-4c3c-894e-8726a3a9335a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:15.397691 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ea96f-22eb-4c3c-894e-8726a3a9335a admin admin] PATH_INFO: `/v3/users/bdfff719bbbe4a1fbfcc6e7f617f62e1` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:15.398954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31ea96f-22eb-4c3c-894e-8726a3a9335a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdfff719bbbe4a1fbfcc6e7f617f62e1)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:15.400908 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31ea96f-22eb-4c3c-894e-8726a3a9335a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:15.450207 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f31ea96f-22eb-4c3c-894e-8726a3a9335a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdfff719bbbe4a1fbfcc6e7f617f62e1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:15.457700 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f31ea96f-22eb-4c3c-894e-8726a3a9335a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdfff719bbbe4a1fbfcc6e7f617f62e1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:15.458399 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f31ea96f-22eb-4c3c-894e-8726a3a9335a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdfff719bbbe4a1fbfcc6e7f617f62e1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:15.465594 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f31ea96f-22eb-4c3c-894e-8726a3a9335a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdfff719bbbe4a1fbfcc6e7f617f62e1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:15.478034 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f31ea96f-22eb-4c3c-894e-8726a3a9335a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdfff719bbbe4a1fbfcc6e7f617f62e1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:15.483884 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1292/2609] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:15 2022] DELETE /identity/v3/users/bdfff719bbbe4a1fbfcc6e7f617f62e1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:15.488346 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd68b3b-5582-4467-b92b-c4ccfd73f10a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:15.488637 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd68b3b-5582-4467-b92b-c4ccfd73f10a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:15.488919 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd68b3b-5582-4467-b92b-c4ccfd73f10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:15.508163 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-6fd68b3b-5582-4467-b92b-c4ccfd73f10a None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:15.509355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6fd68b3b-5582-4467-b92b-c4ccfd73f10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:15.549849 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:15.568886 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1318/2610] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:43:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:15.579267 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ea96f-22eb-4c3c-894e-8726a3a9335a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:15.580690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f31ea96f-22eb-4c3c-894e-8726a3a9335a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:15.594103 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:15.639965 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ea96f-22eb-4c3c-894e-8726a3a9335a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:15.644172 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44671a75-c751-40f8-8915-d076d672f90c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:15.644771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44671a75-c751-40f8-8915-d076d672f90c service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:15.645047 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44671a75-c751-40f8-8915-d076d672f90c service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:15.645317 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44671a75-c751-40f8-8915-d076d672f90c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:15.654116 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44671a75-c751-40f8-8915-d076d672f90c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:15.655827 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44671a75-c751-40f8-8915-d076d672f90c service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:15.664942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1293/2611] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:15.937174 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd68b3b-5582-4467-b92b-c4ccfd73f10a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:15.939228 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6fd68b3b-5582-4467-b92b-c4ccfd73f10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:15.957319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd68b3b-5582-4467-b92b-c4ccfd73f10a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:15.962039 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485ded1-ad94-412e-8952-c29df99d2d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:15.962700 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485ded1-ad94-412e-8952-c29df99d2d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:15.962985 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485ded1-ad94-412e-8952-c29df99d2d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:15.963258 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485ded1-ad94-412e-8952-c29df99d2d1f admin admin] PATH_INFO: `/v3/projects/2d26b3d2008e43e3a277dbf3ca09d34a` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:15.964407 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485ded1-ad94-412e-8952-c29df99d2d1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d26b3d2008e43e3a277dbf3ca09d34a)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:15.966359 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485ded1-ad94-412e-8952-c29df99d2d1f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:16.012840 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7485ded1-ad94-412e-8952-c29df99d2d1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d26b3d2008e43e3a277dbf3ca09d34a', 'request_id': 'req-7485ded1-ad94-412e-8952-c29df99d2d1f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:16.019029 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7485ded1-ad94-412e-8952-c29df99d2d1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d26b3d2008e43e3a277dbf3ca09d34a', 'request_id': 'req-7485ded1-ad94-412e-8952-c29df99d2d1f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:16.024570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7485ded1-ad94-412e-8952-c29df99d2d1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d26b3d2008e43e3a277dbf3ca09d34a', 'request_id': 'req-7485ded1-ad94-412e-8952-c29df99d2d1f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:16.037167 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7485ded1-ad94-412e-8952-c29df99d2d1f admin admin] The token cache is being invalidate because project 2d26b3d2008e43e3a277dbf3ca09d34a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:16.037455 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7485ded1-ad94-412e-8952-c29df99d2d1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:16.038071 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7485ded1-ad94-412e-8952-c29df99d2d1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:16.039568 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1319/2612] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:15 2022] DELETE /identity/v3/projects/2d26b3d2008e43e3a277dbf3ca09d34a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:16.043491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44671a75-c751-40f8-8915-d076d672f90c service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:16.056514 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44671a75-c751-40f8-8915-d076d672f90c service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:16.112407 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57942aee-d6d8-4d4f-9809-f7d4f7b074dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:16.112985 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57942aee-d6d8-4d4f-9809-f7d4f7b074dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:16.113271 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57942aee-d6d8-4d4f-9809-f7d4f7b074dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:16.113572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57942aee-d6d8-4d4f-9809-f7d4f7b074dc admin admin] PATH_INFO: `/v3/users/dac4549def66408f8d12e102a83a33df` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:16.114751 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57942aee-d6d8-4d4f-9809-f7d4f7b074dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=dac4549def66408f8d12e102a83a33df)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:16.116691 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57942aee-d6d8-4d4f-9809-f7d4f7b074dc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:16.163005 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-57942aee-d6d8-4d4f-9809-f7d4f7b074dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dac4549def66408f8d12e102a83a33df'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:16.169821 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-57942aee-d6d8-4d4f-9809-f7d4f7b074dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dac4549def66408f8d12e102a83a33df'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:16.170504 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-57942aee-d6d8-4d4f-9809-f7d4f7b074dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dac4549def66408f8d12e102a83a33df'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:16.178520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-57942aee-d6d8-4d4f-9809-f7d4f7b074dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dac4549def66408f8d12e102a83a33df'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:16.191144 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-57942aee-d6d8-4d4f-9809-f7d4f7b074dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dac4549def66408f8d12e102a83a33df'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:16.196917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1294/2613] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:16 2022] DELETE /identity/v3/users/dac4549def66408f8d12e102a83a33df => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:16.473724 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485ded1-ad94-412e-8952-c29df99d2d1f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:16.475183 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7485ded1-ad94-412e-8952-c29df99d2d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:16.490561 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485ded1-ad94-412e-8952-c29df99d2d1f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:16.548614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd0113-9853-4881-ae57-758f780db341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:16.549245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd0113-9853-4881-ae57-758f780db341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:16.549525 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd0113-9853-4881-ae57-758f780db341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:16.549799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd0113-9853-4881-ae57-758f780db341 admin admin] PATH_INFO: `/v3/projects/40fd89822af7467aa7d33b7fc96423fb` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:16.550938 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cd0113-9853-4881-ae57-758f780db341 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40fd89822af7467aa7d33b7fc96423fb)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:16.552853 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cd0113-9853-4881-ae57-758f780db341 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:16.597938 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-50cd0113-9853-4881-ae57-758f780db341 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40fd89822af7467aa7d33b7fc96423fb', 'request_id': 'req-50cd0113-9853-4881-ae57-758f780db341'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:16.603984 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-50cd0113-9853-4881-ae57-758f780db341 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40fd89822af7467aa7d33b7fc96423fb', 'request_id': 'req-50cd0113-9853-4881-ae57-758f780db341'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:16.610080 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-50cd0113-9853-4881-ae57-758f780db341 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40fd89822af7467aa7d33b7fc96423fb', 'request_id': 'req-50cd0113-9853-4881-ae57-758f780db341'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:16.618101 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:16.625553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-50cd0113-9853-4881-ae57-758f780db341 admin admin] The token cache is being invalidate because project 40fd89822af7467aa7d33b7fc96423fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:16.626086 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-50cd0113-9853-4881-ae57-758f780db341 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:16.627174 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-50cd0113-9853-4881-ae57-758f780db341 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:16.628986 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1320/2614] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:16 2022] DELETE /identity/v3/projects/40fd89822af7467aa7d33b7fc96423fb => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:16.634558 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb237bb9-6a46-41e1-b8c0-21239ceb6de3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:16.634845 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb237bb9-6a46-41e1-b8c0-21239ceb6de3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:16.635148 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb237bb9-6a46-41e1-b8c0-21239ceb6de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:16.655346 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-bb237bb9-6a46-41e1-b8c0-21239ceb6de3 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:16.656756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bb237bb9-6a46-41e1-b8c0-21239ceb6de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:16.686013 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:16.717967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1295/2615] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:16.725669 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a33c79-2fa4-41be-9217-dbe0f5dabb0b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:16.725972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a33c79-2fa4-41be-9217-dbe0f5dabb0b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:16.726257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a33c79-2fa4-41be-9217-dbe0f5dabb0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:16.751452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a3a33c79-2fa4-41be-9217-dbe0f5dabb0b None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:16.752812 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a3a33c79-2fa4-41be-9217-dbe0f5dabb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:16.764347 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1321/2616] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:16.768910 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb237bb9-6a46-41e1-b8c0-21239ceb6de3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:16.778600 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb237bb9-6a46-41e1-b8c0-21239ceb6de3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:16.783392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc02d3ce-4fb3-48b5-b460-60dd6ffc0c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:16.783992 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02d3ce-4fb3-48b5-b460-60dd6ffc0c77 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:16.784276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02d3ce-4fb3-48b5-b460-60dd6ffc0c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:16.784551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02d3ce-4fb3-48b5-b460-60dd6ffc0c77 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:16.785199 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc02d3ce-4fb3-48b5-b460-60dd6ffc0c77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:43:16.785533 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc02d3ce-4fb3-48b5-b460-60dd6ffc0c77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:16.787160 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc02d3ce-4fb3-48b5-b460-60dd6ffc0c77 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:16.799222 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1296/2617] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:43:16.803529 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a33c79-2fa4-41be-9217-dbe0f5dabb0b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:16.813819 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a33c79-2fa4-41be-9217-dbe0f5dabb0b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:16.818432 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a23a8d-e9cd-41e4-84dd-65020f793936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:16.819018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a23a8d-e9cd-41e4-84dd-65020f793936 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:16.819296 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a23a8d-e9cd-41e4-84dd-65020f793936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:16.819568 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a23a8d-e9cd-41e4-84dd-65020f793936 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:16.820191 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a23a8d-e9cd-41e4-84dd-65020f793936 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:16.822051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a23a8d-e9cd-41e4-84dd-65020f793936 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:16.848231 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1322/2618] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:16 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:16.852570 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc02d3ce-4fb3-48b5-b460-60dd6ffc0c77 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:16.863008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc02d3ce-4fb3-48b5-b460-60dd6ffc0c77 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:16.918702 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7baca2f-38c1-4d49-9cd4-b4911dad48df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:16.919303 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7baca2f-38c1-4d49-9cd4-b4911dad48df admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:16.919711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7baca2f-38c1-4d49-9cd4-b4911dad48df admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:16.919878 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7baca2f-38c1-4d49-9cd4-b4911dad48df admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:16.920568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7baca2f-38c1-4d49-9cd4-b4911dad48df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:16.923081 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7baca2f-38c1-4d49-9cd4-b4911dad48df admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:16.944402 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1297/2619] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:16 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:16.948397 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a23a8d-e9cd-41e4-84dd-65020f793936 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:16.958770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a23a8d-e9cd-41e4-84dd-65020f793936 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:16.963205 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3e812-58c2-4559-befa-efaa500ccf57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:16.963793 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3e812-58c2-4559-befa-efaa500ccf57 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:16.964069 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3e812-58c2-4559-befa-efaa500ccf57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:16.964335 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3e812-58c2-4559-befa-efaa500ccf57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:16.964965 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e3e812-58c2-4559-befa-efaa500ccf57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:16.966579 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e3e812-58c2-4559-befa-efaa500ccf57 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:16.976218 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1323/2620] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:16.979852 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7baca2f-38c1-4d49-9cd4-b4911dad48df admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:16.990306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7baca2f-38c1-4d49-9cd4-b4911dad48df admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:16.994853 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadc0758-82c5-4c6b-845a-97c78da6aa88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:16.995436 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc0758-82c5-4c6b-845a-97c78da6aa88 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:16.995718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc0758-82c5-4c6b-845a-97c78da6aa88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:16.995990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc0758-82c5-4c6b-845a-97c78da6aa88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:16.996628 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadc0758-82c5-4c6b-845a-97c78da6aa88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:16.998210 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadc0758-82c5-4c6b-845a-97c78da6aa88 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:17.009170 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1298/2621] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:17.013114 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3e812-58c2-4559-befa-efaa500ccf57 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:17.022838 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3e812-58c2-4559-befa-efaa500ccf57 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:17.027358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd57492a-7a44-4824-b4ad-915690969d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:17.027983 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd57492a-7a44-4824-b4ad-915690969d0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:17.028259 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd57492a-7a44-4824-b4ad-915690969d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:17.028530 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd57492a-7a44-4824-b4ad-915690969d0d admin admin] PATH_INFO: `/v3/projects/3182e278cccc4eedb4941217939cb865/users/c5e0a6ee2b164573ac408c051c6663bd/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:17.050180 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd57492a-7a44-4824-b4ad-915690969d0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3182e278cccc4eedb4941217939cb865, user_id=c5e0a6ee2b164573ac408c051c6663bd, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:17.052276 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd57492a-7a44-4824-b4ad-915690969d0d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:17.064647 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1324/2622] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:17 2022] PUT /identity/v3/projects/3182e278cccc4eedb4941217939cb865/users/c5e0a6ee2b164573ac408c051c6663bd/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:17.075823 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa18724-7ae5-4137-973e-cf2a99572249 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:17.076304 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa18724-7ae5-4137-973e-cf2a99572249 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:17.076411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa18724-7ae5-4137-973e-cf2a99572249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:17.104026 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-daa18724-7ae5-4137-973e-cf2a99572249 None None] MFA Rules not processed for user `c5e0a6ee2b164573ac408c051c6663bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:17.105233 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-daa18724-7ae5-4137-973e-cf2a99572249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:17.168411 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1299/2623] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:43:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:17.173607 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd57492a-7a44-4824-b4ad-915690969d0d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:17.233527 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd57492a-7a44-4824-b4ad-915690969d0d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:17.238730 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6e441-631f-4b98-82d3-f53168754df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:17.239326 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6e441-631f-4b98-82d3-f53168754df8 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:17.239608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6e441-631f-4b98-82d3-f53168754df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:17.239880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6e441-631f-4b98-82d3-f53168754df8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:17.240499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6e441-631f-4b98-82d3-f53168754df8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:17.242407 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6e441-631f-4b98-82d3-f53168754df8 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:17.268053 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1325/2624] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:17 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:17.273085 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa18724-7ae5-4137-973e-cf2a99572249 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:17.284965 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa18724-7ae5-4137-973e-cf2a99572249 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:17.311152 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:17.340490 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f91343-73ee-4231-bc23-42af084da43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:17.341096 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f91343-73ee-4231-bc23-42af084da43c admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:17.341377 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f91343-73ee-4231-bc23-42af084da43c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:17.341653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f91343-73ee-4231-bc23-42af084da43c admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:17.342307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f91343-73ee-4231-bc23-42af084da43c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:17.344707 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f91343-73ee-4231-bc23-42af084da43c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:17.365683 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1300/2625] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:17 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:17.369497 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6e441-631f-4b98-82d3-f53168754df8 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:17.381209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6e441-631f-4b98-82d3-f53168754df8 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:17.385741 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ea3d9-827d-42a5-b74d-686dce624b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:17.386366 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ea3d9-827d-42a5-b74d-686dce624b35 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:17.386669 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ea3d9-827d-42a5-b74d-686dce624b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:17.386942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ea3d9-827d-42a5-b74d-686dce624b35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:17.387585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ea3d9-827d-42a5-b74d-686dce624b35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:17.389154 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ea3d9-827d-42a5-b74d-686dce624b35 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:17.400270 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1326/2626] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:43:17.404710 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f91343-73ee-4231-bc23-42af084da43c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:17.415738 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f91343-73ee-4231-bc23-42af084da43c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:17.420287 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9434cb29-f688-4196-bb5c-1a3331639f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:17.420877 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434cb29-f688-4196-bb5c-1a3331639f2d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:17.421155 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434cb29-f688-4196-bb5c-1a3331639f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:17.421426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434cb29-f688-4196-bb5c-1a3331639f2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:17.422092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9434cb29-f688-4196-bb5c-1a3331639f2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:17.423752 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9434cb29-f688-4196-bb5c-1a3331639f2d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:17.433375 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1301/2627] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:17.437581 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ea3d9-827d-42a5-b74d-686dce624b35 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:17.447567 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ea3d9-827d-42a5-b74d-686dce624b35 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:17.452053 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc9c4a-234c-4e36-84cf-38d5071eae68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:17.452686 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc9c4a-234c-4e36-84cf-38d5071eae68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:17.453474 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc9c4a-234c-4e36-84cf-38d5071eae68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:17.453761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc9c4a-234c-4e36-84cf-38d5071eae68 admin admin] PATH_INFO: `/v3/projects/f5743639047a4571b00ead7055d0b5a9/users/c01038f3c8084d9fb43fc82197b04609/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:17.467717 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc9c4a-234c-4e36-84cf-38d5071eae68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5743639047a4571b00ead7055d0b5a9, user_id=c01038f3c8084d9fb43fc82197b04609, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:17.471038 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc9c4a-234c-4e36-84cf-38d5071eae68 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:17.483187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1327/2628] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:17 2022] PUT /identity/v3/projects/f5743639047a4571b00ead7055d0b5a9/users/c01038f3c8084d9fb43fc82197b04609/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:17.493161 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992fefd-ef83-450c-932b-5eae1d7a72be None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:17.493481 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992fefd-ef83-450c-932b-5eae1d7a72be None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:17.493759 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992fefd-ef83-450c-932b-5eae1d7a72be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:17.521333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d992fefd-ef83-450c-932b-5eae1d7a72be None None] MFA Rules not processed for user `c01038f3c8084d9fb43fc82197b04609`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:17.522560 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d992fefd-ef83-450c-932b-5eae1d7a72be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:17.585739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1302/2629] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:43:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:17.590350 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc9c4a-234c-4e36-84cf-38d5071eae68 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:17.635757 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:17.652713 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc9c4a-234c-4e36-84cf-38d5071eae68 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:17.658441 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910d4bb-a98e-4958-b40e-67e5f31c7528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:17.659042 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910d4bb-a98e-4958-b40e-67e5f31c7528 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:17.659321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910d4bb-a98e-4958-b40e-67e5f31c7528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:17.659594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910d4bb-a98e-4958-b40e-67e5f31c7528 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:17.660237 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910d4bb-a98e-4958-b40e-67e5f31c7528 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:17.662158 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910d4bb-a98e-4958-b40e-67e5f31c7528 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:17.688301 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1328/2630] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:17 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:17.692420 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992fefd-ef83-450c-932b-5eae1d7a72be None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:17.704177 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992fefd-ef83-450c-932b-5eae1d7a72be None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:17.761491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdef653-9b19-4030-a09d-e6988ec458a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:17.762094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdef653-9b19-4030-a09d-e6988ec458a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:17.762412 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdef653-9b19-4030-a09d-e6988ec458a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:17.762756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdef653-9b19-4030-a09d-e6988ec458a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:17.763390 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdef653-9b19-4030-a09d-e6988ec458a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:17.765319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdef653-9b19-4030-a09d-e6988ec458a4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:17.787296 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1303/2631] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:17 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:17.791441 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910d4bb-a98e-4958-b40e-67e5f31c7528 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:17.801982 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910d4bb-a98e-4958-b40e-67e5f31c7528 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:17.806755 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec13fa-240e-4b17-a346-fdaaab5edb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:17.807354 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec13fa-240e-4b17-a346-fdaaab5edb64 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:17.807659 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec13fa-240e-4b17-a346-fdaaab5edb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:17.807936 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec13fa-240e-4b17-a346-fdaaab5edb64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:17.808607 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec13fa-240e-4b17-a346-fdaaab5edb64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:17.810391 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec13fa-240e-4b17-a346-fdaaab5edb64 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:17.821819 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1329/2632] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:17.825782 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdef653-9b19-4030-a09d-e6988ec458a4 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:17.836469 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdef653-9b19-4030-a09d-e6988ec458a4 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:17.841013 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591f327-356e-4ef2-8cb0-1b65dd84681b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:17.841653 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591f327-356e-4ef2-8cb0-1b65dd84681b admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:17.841935 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591f327-356e-4ef2-8cb0-1b65dd84681b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:17.842225 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591f327-356e-4ef2-8cb0-1b65dd84681b admin admin] PATH_INFO: `/v3/projects/2b4de3069b264e32b88ceafe40fe61df/users/3281cffadde549c690b0f6605ab945b4/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:17.856791 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591f327-356e-4ef2-8cb0-1b65dd84681b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b4de3069b264e32b88ceafe40fe61df, user_id=3281cffadde549c690b0f6605ab945b4, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:17.859465 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591f327-356e-4ef2-8cb0-1b65dd84681b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:17.871449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1304/2633] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:17 2022] PUT /identity/v3/projects/2b4de3069b264e32b88ceafe40fe61df/users/3281cffadde549c690b0f6605ab945b4/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:17.875792 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec13fa-240e-4b17-a346-fdaaab5edb64 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:17.885676 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec13fa-240e-4b17-a346-fdaaab5edb64 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:17.945437 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc96f4-717a-4d54-8aec-f447c3cdd807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:17.945993 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc96f4-717a-4d54-8aec-f447c3cdd807 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:17.946333 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc96f4-717a-4d54-8aec-f447c3cdd807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:17.946641 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc96f4-717a-4d54-8aec-f447c3cdd807 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:17.947324 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc96f4-717a-4d54-8aec-f447c3cdd807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:17.949057 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc96f4-717a-4d54-8aec-f447c3cdd807 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:17.959623 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1330/2634] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:17.963759 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591f327-356e-4ef2-8cb0-1b65dd84681b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:17.969219 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:17.975278 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591f327-356e-4ef2-8cb0-1b65dd84681b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:17.980408 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a94802-d98d-4f2e-8019-6e229792a827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:17.981093 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a94802-d98d-4f2e-8019-6e229792a827 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:17.981374 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a94802-d98d-4f2e-8019-6e229792a827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:17.981650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a94802-d98d-4f2e-8019-6e229792a827 admin admin] PATH_INFO: `/v3/projects/2b4de3069b264e32b88ceafe40fe61df/users/3281cffadde549c690b0f6605ab945b4/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:17.983620 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a94802-d98d-4f2e-8019-6e229792a827 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b4de3069b264e32b88ceafe40fe61df, user_id=3281cffadde549c690b0f6605ab945b4, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:17.986299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a94802-d98d-4f2e-8019-6e229792a827 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:17.996342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1305/2635] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:17 2022] PUT /identity/v3/projects/2b4de3069b264e32b88ceafe40fe61df/users/3281cffadde549c690b0f6605ab945b4/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:18.006998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171d38b-8e59-495f-985c-5b9e3adbb6ab None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:18.007313 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171d38b-8e59-495f-985c-5b9e3adbb6ab None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:18.007594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171d38b-8e59-495f-985c-5b9e3adbb6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:18.036483 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7171d38b-8e59-495f-985c-5b9e3adbb6ab None None] MFA Rules not processed for user `3281cffadde549c690b0f6605ab945b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:18.037708 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7171d38b-8e59-495f-985c-5b9e3adbb6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:18.105897 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1331/2636] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:43:18 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:18.117756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a94802-d98d-4f2e-8019-6e229792a827 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:18.119510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b7a94802-d98d-4f2e-8019-6e229792a827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:18.177715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a94802-d98d-4f2e-8019-6e229792a827 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:18.181463 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911870f-a310-4be1-a8a2-097f9c791715 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:18.182059 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911870f-a310-4be1-a8a2-097f9c791715 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:18.182377 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911870f-a310-4be1-a8a2-097f9c791715 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:18.182660 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911870f-a310-4be1-a8a2-097f9c791715 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:18.237869 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911870f-a310-4be1-a8a2-097f9c791715 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:18.239268 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911870f-a310-4be1-a8a2-097f9c791715 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:18.248066 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1306/2637] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:25.007878 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171d38b-8e59-495f-985c-5b9e3adbb6ab None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:25.017457 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171d38b-8e59-495f-985c-5b9e3adbb6ab None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:25.021432 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901de034-4337-4906-a3da-fddf804ddcb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:25.022015 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901de034-4337-4906-a3da-fddf804ddcb1 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:25.022326 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901de034-4337-4906-a3da-fddf804ddcb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:25.022600 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901de034-4337-4906-a3da-fddf804ddcb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:25.060232 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:25.081803 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901de034-4337-4906-a3da-fddf804ddcb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:25.083263 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901de034-4337-4906-a3da-fddf804ddcb1 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:25.091997 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1332/2638] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:25.609790 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911870f-a310-4be1-a8a2-097f9c791715 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:25.618826 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911870f-a310-4be1-a8a2-097f9c791715 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:25.623076 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ccbe53-35e5-4445-bd09-e5ef7501508d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:25.623734 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ccbe53-35e5-4445-bd09-e5ef7501508d service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:25.624246 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ccbe53-35e5-4445-bd09-e5ef7501508d service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:25.624589 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ccbe53-35e5-4445-bd09-e5ef7501508d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:25.634264 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ccbe53-35e5-4445-bd09-e5ef7501508d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:25.635698 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ccbe53-35e5-4445-bd09-e5ef7501508d service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:25.645068 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1307/2639] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:30.582787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901de034-4337-4906-a3da-fddf804ddcb1 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:30.654452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901de034-4337-4906-a3da-fddf804ddcb1 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:30.659193 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc23344-088b-4308-b10f-4f086d85975e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:30.659818 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc23344-088b-4308-b10f-4f086d85975e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:30.660103 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc23344-088b-4308-b10f-4f086d85975e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:30.660377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc23344-088b-4308-b10f-4f086d85975e admin admin] PATH_INFO: `/v3/users/c5e0a6ee2b164573ac408c051c6663bd` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:30.662024 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc23344-088b-4308-b10f-4f086d85975e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5e0a6ee2b164573ac408c051c6663bd)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:30.663969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc23344-088b-4308-b10f-4f086d85975e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:30.722403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8cc23344-088b-4308-b10f-4f086d85975e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5e0a6ee2b164573ac408c051c6663bd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:30.733633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8cc23344-088b-4308-b10f-4f086d85975e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5e0a6ee2b164573ac408c051c6663bd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:30.734513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8cc23344-088b-4308-b10f-4f086d85975e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5e0a6ee2b164573ac408c051c6663bd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:30.735085 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8cc23344-088b-4308-b10f-4f086d85975e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5e0a6ee2b164573ac408c051c6663bd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:30.748433 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8cc23344-088b-4308-b10f-4f086d85975e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5e0a6ee2b164573ac408c051c6663bd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:30.761884 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1333/2640] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:30 2022] DELETE /identity/v3/users/c5e0a6ee2b164573ac408c051c6663bd => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:30.766317 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420f3b2-c3d2-4835-86d9-c7cac1c0b963 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:30.766795 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420f3b2-c3d2-4835-86d9-c7cac1c0b963 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:30.766940 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420f3b2-c3d2-4835-86d9-c7cac1c0b963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:30.787911 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3420f3b2-c3d2-4835-86d9-c7cac1c0b963 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:30.789097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3420f3b2-c3d2-4835-86d9-c7cac1c0b963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:30.846055 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1308/2641] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:43:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:30.857882 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc23344-088b-4308-b10f-4f086d85975e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:30.859751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8cc23344-088b-4308-b10f-4f086d85975e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:30.937497 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc23344-088b-4308-b10f-4f086d85975e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:30.941245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68524be3-95bc-47f4-9ac5-743182eb9f56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:30.941847 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68524be3-95bc-47f4-9ac5-743182eb9f56 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:30.942129 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68524be3-95bc-47f4-9ac5-743182eb9f56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:30.942435 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68524be3-95bc-47f4-9ac5-743182eb9f56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:30.951328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68524be3-95bc-47f4-9ac5-743182eb9f56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:30.952746 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68524be3-95bc-47f4-9ac5-743182eb9f56 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:30.957718 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:30.963739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1334/2642] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:31.227113 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3420f3b2-c3d2-4835-86d9-c7cac1c0b963 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:31.228561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3420f3b2-c3d2-4835-86d9-c7cac1c0b963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:31.241187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3420f3b2-c3d2-4835-86d9-c7cac1c0b963 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:31.246631 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9d1af-9897-4ec9-a174-3e865d000237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:31.247301 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9d1af-9897-4ec9-a174-3e865d000237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:31.247590 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9d1af-9897-4ec9-a174-3e865d000237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:31.247926 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9d1af-9897-4ec9-a174-3e865d000237 admin admin] PATH_INFO: `/v3/projects/3182e278cccc4eedb4941217939cb865` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:31.249122 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c9d1af-9897-4ec9-a174-3e865d000237 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3182e278cccc4eedb4941217939cb865)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:31.251050 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c9d1af-9897-4ec9-a174-3e865d000237 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:31.283734 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:31.301643 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-79c9d1af-9897-4ec9-a174-3e865d000237 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3182e278cccc4eedb4941217939cb865', 'request_id': 'req-79c9d1af-9897-4ec9-a174-3e865d000237'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.307820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-79c9d1af-9897-4ec9-a174-3e865d000237 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3182e278cccc4eedb4941217939cb865', 'request_id': 'req-79c9d1af-9897-4ec9-a174-3e865d000237'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.315291 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-79c9d1af-9897-4ec9-a174-3e865d000237 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3182e278cccc4eedb4941217939cb865', 'request_id': 'req-79c9d1af-9897-4ec9-a174-3e865d000237'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.326443 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-79c9d1af-9897-4ec9-a174-3e865d000237 admin admin] The token cache is being invalidate because project 3182e278cccc4eedb4941217939cb865 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:31.326735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-79c9d1af-9897-4ec9-a174-3e865d000237 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.327361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-79c9d1af-9897-4ec9-a174-3e865d000237 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.328838 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1309/2643] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:31 2022] DELETE /identity/v3/projects/3182e278cccc4eedb4941217939cb865 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:31.332793 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68524be3-95bc-47f4-9ac5-743182eb9f56 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:31.342958 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68524be3-95bc-47f4-9ac5-743182eb9f56 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:31.398942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a783c6-e4a2-42ff-b7e9-cb76a73d1365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:31.399573 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a783c6-e4a2-42ff-b7e9-cb76a73d1365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:31.399850 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a783c6-e4a2-42ff-b7e9-cb76a73d1365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:31.400123 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a783c6-e4a2-42ff-b7e9-cb76a73d1365 admin admin] PATH_INFO: `/v3/users/c01038f3c8084d9fb43fc82197b04609` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:31.401227 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a783c6-e4a2-42ff-b7e9-cb76a73d1365 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c01038f3c8084d9fb43fc82197b04609)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:31.403176 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a783c6-e4a2-42ff-b7e9-cb76a73d1365 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:31.451685 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-73a783c6-e4a2-42ff-b7e9-cb76a73d1365 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c01038f3c8084d9fb43fc82197b04609'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.459226 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-73a783c6-e4a2-42ff-b7e9-cb76a73d1365 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c01038f3c8084d9fb43fc82197b04609'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.459875 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-73a783c6-e4a2-42ff-b7e9-cb76a73d1365 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c01038f3c8084d9fb43fc82197b04609'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.460470 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-73a783c6-e4a2-42ff-b7e9-cb76a73d1365 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c01038f3c8084d9fb43fc82197b04609'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.472133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-73a783c6-e4a2-42ff-b7e9-cb76a73d1365 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c01038f3c8084d9fb43fc82197b04609'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.484571 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1335/2644] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:31 2022] DELETE /identity/v3/users/c01038f3c8084d9fb43fc82197b04609 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:31.754169 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9d1af-9897-4ec9-a174-3e865d000237 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:31.756233 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-79c9d1af-9897-4ec9-a174-3e865d000237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:31.772089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9d1af-9897-4ec9-a174-3e865d000237 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:31.830248 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80493d4f-e0d2-4b49-a399-fb381b0d2fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:31.830882 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80493d4f-e0d2-4b49-a399-fb381b0d2fbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:31.831166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80493d4f-e0d2-4b49-a399-fb381b0d2fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:31.831441 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80493d4f-e0d2-4b49-a399-fb381b0d2fbb admin admin] PATH_INFO: `/v3/projects/f5743639047a4571b00ead7055d0b5a9` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:31.832546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80493d4f-e0d2-4b49-a399-fb381b0d2fbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5743639047a4571b00ead7055d0b5a9)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:31.834541 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80493d4f-e0d2-4b49-a399-fb381b0d2fbb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:31.880478 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-80493d4f-e0d2-4b49-a399-fb381b0d2fbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5743639047a4571b00ead7055d0b5a9', 'request_id': 'req-80493d4f-e0d2-4b49-a399-fb381b0d2fbb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.886540 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-80493d4f-e0d2-4b49-a399-fb381b0d2fbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5743639047a4571b00ead7055d0b5a9', 'request_id': 'req-80493d4f-e0d2-4b49-a399-fb381b0d2fbb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.893868 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-80493d4f-e0d2-4b49-a399-fb381b0d2fbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5743639047a4571b00ead7055d0b5a9', 'request_id': 'req-80493d4f-e0d2-4b49-a399-fb381b0d2fbb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.904573 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-80493d4f-e0d2-4b49-a399-fb381b0d2fbb admin admin] The token cache is being invalidate because project f5743639047a4571b00ead7055d0b5a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:31.904864 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-80493d4f-e0d2-4b49-a399-fb381b0d2fbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.905502 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-80493d4f-e0d2-4b49-a399-fb381b0d2fbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.907014 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1310/2645] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:31 2022] DELETE /identity/v3/projects/f5743639047a4571b00ead7055d0b5a9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:31.911764 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a783c6-e4a2-42ff-b7e9-cb76a73d1365 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:31.913147 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-73a783c6-e4a2-42ff-b7e9-cb76a73d1365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:31.925820 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a783c6-e4a2-42ff-b7e9-cb76a73d1365 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:31.982165 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389cfed-5fdf-4bd9-8453-da8b2e0a7e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:31.982763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389cfed-5fdf-4bd9-8453-da8b2e0a7e19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:31.983064 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389cfed-5fdf-4bd9-8453-da8b2e0a7e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:31.983323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389cfed-5fdf-4bd9-8453-da8b2e0a7e19 admin admin] PATH_INFO: `/v3/users/3281cffadde549c690b0f6605ab945b4` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:31.984451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389cfed-5fdf-4bd9-8453-da8b2e0a7e19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3281cffadde549c690b0f6605ab945b4)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:31.986421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389cfed-5fdf-4bd9-8453-da8b2e0a7e19 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:32.008004 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:32.034048 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-9389cfed-5fdf-4bd9-8453-da8b2e0a7e19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3281cffadde549c690b0f6605ab945b4'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:32.041235 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-9389cfed-5fdf-4bd9-8453-da8b2e0a7e19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3281cffadde549c690b0f6605ab945b4'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:32.041877 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-9389cfed-5fdf-4bd9-8453-da8b2e0a7e19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3281cffadde549c690b0f6605ab945b4'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:32.042526 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-9389cfed-5fdf-4bd9-8453-da8b2e0a7e19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3281cffadde549c690b0f6605ab945b4'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:32.055094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-9389cfed-5fdf-4bd9-8453-da8b2e0a7e19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3281cffadde549c690b0f6605ab945b4'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:32.068010 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1336/2646] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:31 2022] DELETE /identity/v3/users/3281cffadde549c690b0f6605ab945b4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:32.333497 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80493d4f-e0d2-4b49-a399-fb381b0d2fbb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:32.334971 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-80493d4f-e0d2-4b49-a399-fb381b0d2fbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:32.350923 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80493d4f-e0d2-4b49-a399-fb381b0d2fbb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:32.394142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:32.412138 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c27fc26-a807-46b5-a066-07c1d06a5b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:32.412771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27fc26-a807-46b5-a066-07c1d06a5b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:32.413053 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27fc26-a807-46b5-a066-07c1d06a5b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:32.413328 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27fc26-a807-46b5-a066-07c1d06a5b3b admin admin] PATH_INFO: `/v3/projects/2b4de3069b264e32b88ceafe40fe61df` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:32.414485 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c27fc26-a807-46b5-a066-07c1d06a5b3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b4de3069b264e32b88ceafe40fe61df)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:32.416391 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c27fc26-a807-46b5-a066-07c1d06a5b3b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:32.462375 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9c27fc26-a807-46b5-a066-07c1d06a5b3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b4de3069b264e32b88ceafe40fe61df', 'request_id': 'req-9c27fc26-a807-46b5-a066-07c1d06a5b3b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:32.468340 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9c27fc26-a807-46b5-a066-07c1d06a5b3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b4de3069b264e32b88ceafe40fe61df', 'request_id': 'req-9c27fc26-a807-46b5-a066-07c1d06a5b3b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:32.476761 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9c27fc26-a807-46b5-a066-07c1d06a5b3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b4de3069b264e32b88ceafe40fe61df', 'request_id': 'req-9c27fc26-a807-46b5-a066-07c1d06a5b3b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:32.487378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9c27fc26-a807-46b5-a066-07c1d06a5b3b admin admin] The token cache is being invalidate because project 2b4de3069b264e32b88ceafe40fe61df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:32.487669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9c27fc26-a807-46b5-a066-07c1d06a5b3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:32.488291 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9c27fc26-a807-46b5-a066-07c1d06a5b3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:32.489756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1311/2647] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:32 2022] DELETE /identity/v3/projects/2b4de3069b264e32b88ceafe40fe61df => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:32.494548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f9b91-9e6a-4cc6-b771-4bd70c492b2c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:32.494836 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f9b91-9e6a-4cc6-b771-4bd70c492b2c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:32.495117 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f9b91-9e6a-4cc6-b771-4bd70c492b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:32.516657 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-c17f9b91-9e6a-4cc6-b771-4bd70c492b2c None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:32.517778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c17f9b91-9e6a-4cc6-b771-4bd70c492b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:32.576498 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1337/2648] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:32.584722 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118f9c2-fd48-4c2a-80e8-af446d6d78d1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:32.585011 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118f9c2-fd48-4c2a-80e8-af446d6d78d1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:32.585284 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118f9c2-fd48-4c2a-80e8-af446d6d78d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:32.605276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a118f9c2-fd48-4c2a-80e8-af446d6d78d1 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:32.606520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a118f9c2-fd48-4c2a-80e8-af446d6d78d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:32.617330 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1312/2649] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:32.621513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f9b91-9e6a-4cc6-b771-4bd70c492b2c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:32.632214 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f9b91-9e6a-4cc6-b771-4bd70c492b2c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:32.636792 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155af6ce-5408-43bd-bc4f-1d44b0d42efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:32.637385 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155af6ce-5408-43bd-bc4f-1d44b0d42efe admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:32.637665 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155af6ce-5408-43bd-bc4f-1d44b0d42efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:32.637938 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155af6ce-5408-43bd-bc4f-1d44b0d42efe admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:32.638631 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155af6ce-5408-43bd-bc4f-1d44b0d42efe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:43:32.638938 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155af6ce-5408-43bd-bc4f-1d44b0d42efe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:32.640515 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155af6ce-5408-43bd-bc4f-1d44b0d42efe admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:32.652740 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1338/2650] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:43:32.657022 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a118f9c2-fd48-4c2a-80e8-af446d6d78d1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:32.667338 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a118f9c2-fd48-4c2a-80e8-af446d6d78d1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:32.672098 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cddbe2f-9365-4f03-bbc0-aeb24d799bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:32.672689 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddbe2f-9365-4f03-bbc0-aeb24d799bcd admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:32.672968 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddbe2f-9365-4f03-bbc0-aeb24d799bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:32.673237 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddbe2f-9365-4f03-bbc0-aeb24d799bcd admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:32.673869 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cddbe2f-9365-4f03-bbc0-aeb24d799bcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:32.675789 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cddbe2f-9365-4f03-bbc0-aeb24d799bcd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:32.703155 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1313/2651] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:32 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:32.706956 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155af6ce-5408-43bd-bc4f-1d44b0d42efe admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:32.717649 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155af6ce-5408-43bd-bc4f-1d44b0d42efe admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:32.773080 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e42037-270d-4184-b007-8aa801e7f7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:32.773603 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e42037-270d-4184-b007-8aa801e7f7ce admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:32.773878 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e42037-270d-4184-b007-8aa801e7f7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:32.774146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e42037-270d-4184-b007-8aa801e7f7ce admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:32.774802 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e42037-270d-4184-b007-8aa801e7f7ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:32.776691 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e42037-270d-4184-b007-8aa801e7f7ce admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:32.801712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1339/2652] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:32 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:43:32.802866 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cddbe2f-9365-4f03-bbc0-aeb24d799bcd admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:32.816273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cddbe2f-9365-4f03-bbc0-aeb24d799bcd admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:32.821512 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c110c0a-7477-4586-a5d9-6497e3dfb09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:32.822107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c110c0a-7477-4586-a5d9-6497e3dfb09e admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:32.822420 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c110c0a-7477-4586-a5d9-6497e3dfb09e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:32.822811 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c110c0a-7477-4586-a5d9-6497e3dfb09e admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:32.823456 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c110c0a-7477-4586-a5d9-6497e3dfb09e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:32.825047 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c110c0a-7477-4586-a5d9-6497e3dfb09e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:32.839182 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1314/2653] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:32.842673 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e42037-270d-4184-b007-8aa801e7f7ce admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:32.852763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e42037-270d-4184-b007-8aa801e7f7ce admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:32.857421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4efca3-ca23-4e51-b7b4-9bf8b01f5ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:32.858004 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4efca3-ca23-4e51-b7b4-9bf8b01f5ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:32.858312 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4efca3-ca23-4e51-b7b4-9bf8b01f5ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:32.858582 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4efca3-ca23-4e51-b7b4-9bf8b01f5ad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:32.859216 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4efca3-ca23-4e51-b7b4-9bf8b01f5ad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:32.860776 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4efca3-ca23-4e51-b7b4-9bf8b01f5ad7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:32.872627 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1340/2654] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:32.876584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c110c0a-7477-4586-a5d9-6497e3dfb09e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:32.887417 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c110c0a-7477-4586-a5d9-6497e3dfb09e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:32.896080 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440b5f5a-eccd-435a-9f98-b493b9bf247d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:32.896733 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b5f5a-eccd-435a-9f98-b493b9bf247d admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:32.897017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b5f5a-eccd-435a-9f98-b493b9bf247d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:32.897296 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b5f5a-eccd-435a-9f98-b493b9bf247d admin admin] PATH_INFO: `/v3/projects/ca311e408a714ede8d413375ca68eb67/users/3fbb843ada72400e9d9f06521c2a61ab/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:32.913174 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440b5f5a-eccd-435a-9f98-b493b9bf247d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca311e408a714ede8d413375ca68eb67, user_id=3fbb843ada72400e9d9f06521c2a61ab, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:32.915912 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440b5f5a-eccd-435a-9f98-b493b9bf247d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:32.928386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1315/2655] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:32 2022] PUT /identity/v3/projects/ca311e408a714ede8d413375ca68eb67/users/3fbb843ada72400e9d9f06521c2a61ab/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:32.933385 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d0c13-39d6-4930-b9e1-2fdcc9cee18f None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:32.933851 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d0c13-39d6-4930-b9e1-2fdcc9cee18f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:32.934124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d0c13-39d6-4930-b9e1-2fdcc9cee18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:32.942215 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:32.954295 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f50d0c13-39d6-4930-b9e1-2fdcc9cee18f None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:32.955502 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f50d0c13-39d6-4930-b9e1-2fdcc9cee18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:33.012773 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1341/2656] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:33.023420 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440b5f5a-eccd-435a-9f98-b493b9bf247d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:33.025056 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-440b5f5a-eccd-435a-9f98-b493b9bf247d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:33.082342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440b5f5a-eccd-435a-9f98-b493b9bf247d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:33.085938 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fee25f-b707-465e-925b-85632b0537cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:33.086584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fee25f-b707-465e-925b-85632b0537cb service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:33.086864 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fee25f-b707-465e-925b-85632b0537cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:33.087134 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fee25f-b707-465e-925b-85632b0537cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:33.095757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fee25f-b707-465e-925b-85632b0537cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:33.097164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fee25f-b707-465e-925b-85632b0537cb service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:33.106343 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1316/2657] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:33.722172 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce946d65-c119-4b3d-9679-9a5a121f8882 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:33.722496 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce946d65-c119-4b3d-9679-9a5a121f8882 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:33.722778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce946d65-c119-4b3d-9679-9a5a121f8882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:33.753752 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ce946d65-c119-4b3d-9679-9a5a121f8882 None None] MFA Rules not processed for user `3fbb843ada72400e9d9f06521c2a61ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:33.755692 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ce946d65-c119-4b3d-9679-9a5a121f8882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:33.833924 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1342/2658] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:43:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:33.846143 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fee25f-b707-465e-925b-85632b0537cb service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:33.852149 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d4fee25f-b707-465e-925b-85632b0537cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:33.872442 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:33.921065 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fee25f-b707-465e-925b-85632b0537cb service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:33.924857 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab03094-51c2-48da-a585-63a31eea6d8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:33.925470 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab03094-51c2-48da-a585-63a31eea6d8e service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:33.925758 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab03094-51c2-48da-a585-63a31eea6d8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:33.926031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab03094-51c2-48da-a585-63a31eea6d8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:33.934945 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab03094-51c2-48da-a585-63a31eea6d8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:33.936382 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab03094-51c2-48da-a585-63a31eea6d8e service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:33.945801 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1317/2659] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:35.260358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce946d65-c119-4b3d-9679-9a5a121f8882 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:35.269896 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce946d65-c119-4b3d-9679-9a5a121f8882 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:35.274117 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881421fa-8d05-4711-959a-556ae6bbe06c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:35.274766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881421fa-8d05-4711-959a-556ae6bbe06c service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:35.275050 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881421fa-8d05-4711-959a-556ae6bbe06c service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:35.275327 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881421fa-8d05-4711-959a-556ae6bbe06c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:35.277077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-881421fa-8d05-4711-959a-556ae6bbe06c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:35.356403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881421fa-8d05-4711-959a-556ae6bbe06c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:35.357849 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881421fa-8d05-4711-959a-556ae6bbe06c service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:35.376151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1343/2660] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:35 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:37.402879 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd0f61-e2d6-4860-8e2b-dc6d438a8684 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:37.403413 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd0f61-e2d6-4860-8e2b-dc6d438a8684 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:37.403413 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd0f61-e2d6-4860-8e2b-dc6d438a8684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:37.422426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-93dd0f61-e2d6-4860-8e2b-dc6d438a8684 None None] MFA Rules not processed for user `3fbb843ada72400e9d9f06521c2a61ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:37.423611 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-93dd0f61-e2d6-4860-8e2b-dc6d438a8684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:37.430321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1318/2661] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:43:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:37.679786 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881421fa-8d05-4711-959a-556ae6bbe06c service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:37.681226 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-881421fa-8d05-4711-959a-556ae6bbe06c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:37.689024 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:37.693785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881421fa-8d05-4711-959a-556ae6bbe06c service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:37.697745 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95937712-a7e2-4a93-96aa-7dbca7ef0b02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:37.698472 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95937712-a7e2-4a93-96aa-7dbca7ef0b02 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:37.698759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95937712-a7e2-4a93-96aa-7dbca7ef0b02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:37.699036 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95937712-a7e2-4a93-96aa-7dbca7ef0b02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:37.700721 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-95937712-a7e2-4a93-96aa-7dbca7ef0b02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:37.710421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95937712-a7e2-4a93-96aa-7dbca7ef0b02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:37.711840 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95937712-a7e2-4a93-96aa-7dbca7ef0b02 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:37.719751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1344/2662] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:38.067740 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd0f61-e2d6-4860-8e2b-dc6d438a8684 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:38.072880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-93dd0f61-e2d6-4860-8e2b-dc6d438a8684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:38.192475 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd0f61-e2d6-4860-8e2b-dc6d438a8684 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:38.198373 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7fd23-cbb1-466f-9c79-ec4474de910d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12302cdb2854b0684179647a6dadf6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:38.199125 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7fd23-cbb1-466f-9c79-ec4474de910d service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:38.199809 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7fd23-cbb1-466f-9c79-ec4474de910d service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:38.199809 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7fd23-cbb1-466f-9c79-ec4474de910d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:38.202532 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ceb7fd23-cbb1-466f-9c79-ec4474de910d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:38.218361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7fd23-cbb1-466f-9c79-ec4474de910d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:38.219958 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7fd23-cbb1-466f-9c79-ec4474de910d service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:38.237183 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1319/2663] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:39.060513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95937712-a7e2-4a93-96aa-7dbca7ef0b02 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:39.062133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-95937712-a7e2-4a93-96aa-7dbca7ef0b02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:39.155209 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95937712-a7e2-4a93-96aa-7dbca7ef0b02 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:39.165972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fbdfbb-86a6-45d2-b1e3-3f84c12af350 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:39.166788 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fbdfbb-86a6-45d2-b1e3-3f84c12af350 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:39.167163 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fbdfbb-86a6-45d2-b1e3-3f84c12af350 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:39.167639 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fbdfbb-86a6-45d2-b1e3-3f84c12af350 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:39.169428 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a1fbdfbb-86a6-45d2-b1e3-3f84c12af350 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:39.184258 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fbdfbb-86a6-45d2-b1e3-3f84c12af350 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:39.186081 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fbdfbb-86a6-45d2-b1e3-3f84c12af350 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:39.199501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1345/2664] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:43:41.687103 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7fd23-cbb1-466f-9c79-ec4474de910d service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:41.689652 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ceb7fd23-cbb1-466f-9c79-ec4474de910d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:41.704954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7fd23-cbb1-466f-9c79-ec4474de910d service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:41.710628 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6047fc-4ded-4ce3-8339-da6d79a1b74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:41.711537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6047fc-4ded-4ce3-8339-da6d79a1b74a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:41.712075 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6047fc-4ded-4ce3-8339-da6d79a1b74a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:41.712417 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6047fc-4ded-4ce3-8339-da6d79a1b74a admin admin] PATH_INFO: `/v3/users/a7eb999b869048479693b4aaee88410f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:41.713902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6047fc-4ded-4ce3-8339-da6d79a1b74a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7eb999b869048479693b4aaee88410f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:41.716004 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6047fc-4ded-4ce3-8339-da6d79a1b74a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:41.786037 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ee6047fc-4ded-4ce3-8339-da6d79a1b74a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7eb999b869048479693b4aaee88410f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:41.794994 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ee6047fc-4ded-4ce3-8339-da6d79a1b74a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7eb999b869048479693b4aaee88410f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:41.795746 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ee6047fc-4ded-4ce3-8339-da6d79a1b74a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7eb999b869048479693b4aaee88410f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:41.806497 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ee6047fc-4ded-4ce3-8339-da6d79a1b74a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7eb999b869048479693b4aaee88410f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:41.821449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ee6047fc-4ded-4ce3-8339-da6d79a1b74a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7eb999b869048479693b4aaee88410f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:41.828008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1320/2665] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:41 2022] DELETE /identity/v3/users/a7eb999b869048479693b4aaee88410f => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:41.832697 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00df23-92e1-4dcc-872b-9e73f6a17a67 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:41.833186 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00df23-92e1-4dcc-872b-9e73f6a17a67 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:41.833355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00df23-92e1-4dcc-872b-9e73f6a17a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:41.858679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-0e00df23-92e1-4dcc-872b-9e73f6a17a67 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:41.859901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0e00df23-92e1-4dcc-872b-9e73f6a17a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:41.925772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1346/2666] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:41.933032 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6047fc-4ded-4ce3-8339-da6d79a1b74a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:41.934591 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ee6047fc-4ded-4ce3-8339-da6d79a1b74a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:41.949569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:42.000622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6047fc-4ded-4ce3-8339-da6d79a1b74a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:42.004238 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535edea7-9e34-4a9d-b442-b27713faee34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:42.004842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535edea7-9e34-4a9d-b442-b27713faee34 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:42.005117 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535edea7-9e34-4a9d-b442-b27713faee34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:42.005390 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535edea7-9e34-4a9d-b442-b27713faee34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:42.014521 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535edea7-9e34-4a9d-b442-b27713faee34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:42.015949 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535edea7-9e34-4a9d-b442-b27713faee34 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:42.026276 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1321/2667] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:42.313228 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00df23-92e1-4dcc-872b-9e73f6a17a67 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:42.315155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0e00df23-92e1-4dcc-872b-9e73f6a17a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:42.329711 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00df23-92e1-4dcc-872b-9e73f6a17a67 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:42.335784 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efba9d-e098-4d1d-9ed0-00243a40a0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:42.336519 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efba9d-e098-4d1d-9ed0-00243a40a0a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:42.336970 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efba9d-e098-4d1d-9ed0-00243a40a0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:42.337246 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efba9d-e098-4d1d-9ed0-00243a40a0a9 admin admin] PATH_INFO: `/v3/projects/aa3686c0f91d40a1b8b0e1af5b9e93f9` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:42.338461 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efba9d-e098-4d1d-9ed0-00243a40a0a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa3686c0f91d40a1b8b0e1af5b9e93f9)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:42.340540 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efba9d-e098-4d1d-9ed0-00243a40a0a9 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:42.392467 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-69efba9d-e098-4d1d-9ed0-00243a40a0a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa3686c0f91d40a1b8b0e1af5b9e93f9', 'request_id': 'req-69efba9d-e098-4d1d-9ed0-00243a40a0a9'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:42.399210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-69efba9d-e098-4d1d-9ed0-00243a40a0a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa3686c0f91d40a1b8b0e1af5b9e93f9', 'request_id': 'req-69efba9d-e098-4d1d-9ed0-00243a40a0a9'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:42.404731 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-69efba9d-e098-4d1d-9ed0-00243a40a0a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa3686c0f91d40a1b8b0e1af5b9e93f9', 'request_id': 'req-69efba9d-e098-4d1d-9ed0-00243a40a0a9'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:42.413785 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:42.418495 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-69efba9d-e098-4d1d-9ed0-00243a40a0a9 admin admin] The token cache is being invalidate because project aa3686c0f91d40a1b8b0e1af5b9e93f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:42.419047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-69efba9d-e098-4d1d-9ed0-00243a40a0a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:42.419878 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-69efba9d-e098-4d1d-9ed0-00243a40a0a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:42.421750 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1347/2668] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:42 2022] DELETE /identity/v3/projects/aa3686c0f91d40a1b8b0e1af5b9e93f9 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:44.809562 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535edea7-9e34-4a9d-b442-b27713faee34 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:44.818478 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535edea7-9e34-4a9d-b442-b27713faee34 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:44.870778 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a5472-e339-42eb-a51a-125af9f5b180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:44.871454 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a5472-e339-42eb-a51a-125af9f5b180 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:44.871713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a5472-e339-42eb-a51a-125af9f5b180 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:44.871996 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a5472-e339-42eb-a51a-125af9f5b180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:44.873780 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-612a5472-e339-42eb-a51a-125af9f5b180 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:44.946204 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a5472-e339-42eb-a51a-125af9f5b180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:44.947740 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a5472-e339-42eb-a51a-125af9f5b180 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:44.958296 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1322/2669] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:48.192453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efba9d-e098-4d1d-9ed0-00243a40a0a9 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:48.194637 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-69efba9d-e098-4d1d-9ed0-00243a40a0a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:48.208810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efba9d-e098-4d1d-9ed0-00243a40a0a9 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:48.213421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6aa6d4-f5c3-4757-93e0-4687b31f7b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:48.213998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6aa6d4-f5c3-4757-93e0-4687b31f7b41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:48.214306 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6aa6d4-f5c3-4757-93e0-4687b31f7b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:48.214582 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6aa6d4-f5c3-4757-93e0-4687b31f7b41 admin admin] PATH_INFO: `/v3/users/a328e946d6d4441ca7ecbb3a20d2a611` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:48.215693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6aa6d4-f5c3-4757-93e0-4687b31f7b41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a328e946d6d4441ca7ecbb3a20d2a611)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:48.217646 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6aa6d4-f5c3-4757-93e0-4687b31f7b41 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:48.265946 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8f6aa6d4-f5c3-4757-93e0-4687b31f7b41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a328e946d6d4441ca7ecbb3a20d2a611'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:48.273149 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8f6aa6d4-f5c3-4757-93e0-4687b31f7b41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a328e946d6d4441ca7ecbb3a20d2a611'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:48.273818 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8f6aa6d4-f5c3-4757-93e0-4687b31f7b41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a328e946d6d4441ca7ecbb3a20d2a611'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:48.274461 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8f6aa6d4-f5c3-4757-93e0-4687b31f7b41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a328e946d6d4441ca7ecbb3a20d2a611'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:48.286372 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8f6aa6d4-f5c3-4757-93e0-4687b31f7b41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a328e946d6d4441ca7ecbb3a20d2a611'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:48.300117 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1348/2670] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:48 2022] DELETE /identity/v3/users/a328e946d6d4441ca7ecbb3a20d2a611 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:48.486161 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a5472-e339-42eb-a51a-125af9f5b180 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:48.501289 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a5472-e339-42eb-a51a-125af9f5b180 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:48.609949 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee84da-cef3-4b9e-a92d-6033c82950b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:48.610783 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee84da-cef3-4b9e-a92d-6033c82950b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:48.611352 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee84da-cef3-4b9e-a92d-6033c82950b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:48.611699 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee84da-cef3-4b9e-a92d-6033c82950b4 admin admin] PATH_INFO: `/v3/projects/10f3ba5ae75246ec9237a7e2305ae32a` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:48.613130 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ee84da-cef3-4b9e-a92d-6033c82950b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10f3ba5ae75246ec9237a7e2305ae32a)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:48.615881 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ee84da-cef3-4b9e-a92d-6033c82950b4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:48.624755 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:48.671319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-34ee84da-cef3-4b9e-a92d-6033c82950b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10f3ba5ae75246ec9237a7e2305ae32a', 'request_id': 'req-34ee84da-cef3-4b9e-a92d-6033c82950b4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:48.678857 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6aa6d4-f5c3-4757-93e0-4687b31f7b41 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:48.680243 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-34ee84da-cef3-4b9e-a92d-6033c82950b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10f3ba5ae75246ec9237a7e2305ae32a', 'request_id': 'req-34ee84da-cef3-4b9e-a92d-6033c82950b4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:48.680459 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8f6aa6d4-f5c3-4757-93e0-4687b31f7b41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:48.688814 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-34ee84da-cef3-4b9e-a92d-6033c82950b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10f3ba5ae75246ec9237a7e2305ae32a', 'request_id': 'req-34ee84da-cef3-4b9e-a92d-6033c82950b4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:48.700866 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-34ee84da-cef3-4b9e-a92d-6033c82950b4 admin admin] The token cache is being invalidate because project 10f3ba5ae75246ec9237a7e2305ae32a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:48.701248 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-34ee84da-cef3-4b9e-a92d-6033c82950b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:48.702002 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-34ee84da-cef3-4b9e-a92d-6033c82950b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:48.703845 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1323/2671] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:48 2022] DELETE /identity/v3/projects/10f3ba5ae75246ec9237a7e2305ae32a => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:48.709917 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93c42a-517f-4cb3-a8f7-0432ea8c8801 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:48.710222 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93c42a-517f-4cb3-a8f7-0432ea8c8801 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:48.710507 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93c42a-517f-4cb3-a8f7-0432ea8c8801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:48.734296 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-de93c42a-517f-4cb3-a8f7-0432ea8c8801 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:48.735457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-de93c42a-517f-4cb3-a8f7-0432ea8c8801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:48.766534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6aa6d4-f5c3-4757-93e0-4687b31f7b41 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:48.796546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1324/2672] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:43:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:48.804252 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8f605-c6bb-4cb4-9dbc-db0467188eeb None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:48.804534 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8f605-c6bb-4cb4-9dbc-db0467188eeb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:48.804802 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8f605-c6bb-4cb4-9dbc-db0467188eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:48.823330 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e0a8f605-c6bb-4cb4-9dbc-db0467188eeb None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:48.824510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e0a8f605-c6bb-4cb4-9dbc-db0467188eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:48.832262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1325/2673] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:43:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:48.832639 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440a36c-c072-40cc-8e1a-1e2cbad9031c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:48.833258 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440a36c-c072-40cc-8e1a-1e2cbad9031c service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:48.833542 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440a36c-c072-40cc-8e1a-1e2cbad9031c service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:48.833818 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440a36c-c072-40cc-8e1a-1e2cbad9031c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:48.836133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7440a36c-c072-40cc-8e1a-1e2cbad9031c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:48.836737 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8f605-c6bb-4cb4-9dbc-db0467188eeb None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:48.845446 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8f605-c6bb-4cb4-9dbc-db0467188eeb None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:48.845876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:48.849478 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7440a36c-c072-40cc-8e1a-1e2cbad9031c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:48.851026 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46150528-fe57-4b67-923e-b0900ec182a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:48.851607 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46150528-fe57-4b67-923e-b0900ec182a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:48.851910 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46150528-fe57-4b67-923e-b0900ec182a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:48.852048 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7440a36c-c072-40cc-8e1a-1e2cbad9031c service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:48.852185 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46150528-fe57-4b67-923e-b0900ec182a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:48.852807 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46150528-fe57-4b67-923e-b0900ec182a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:43:48.853110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46150528-fe57-4b67-923e-b0900ec182a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:48.854685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46150528-fe57-4b67-923e-b0900ec182a9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:48.864328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1349/2674] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:43:48.867007 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1326/2675] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:43:48.871105 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440a36c-c072-40cc-8e1a-1e2cbad9031c service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:48.881370 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440a36c-c072-40cc-8e1a-1e2cbad9031c service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:48.885980 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295b524-55fa-47ce-9697-14d983febd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:48.886677 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295b524-55fa-47ce-9697-14d983febd27 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:48.887019 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295b524-55fa-47ce-9697-14d983febd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:48.887294 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295b524-55fa-47ce-9697-14d983febd27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:48.887925 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295b524-55fa-47ce-9697-14d983febd27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:48.889786 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295b524-55fa-47ce-9697-14d983febd27 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:48.919191 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1350/2676] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:48 2022] POST /identity/v3/projects => generated 405 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:48.923315 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46150528-fe57-4b67-923e-b0900ec182a9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:48.933443 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46150528-fe57-4b67-923e-b0900ec182a9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:48.990804 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f50588-303b-4c5f-8c8f-17aa98b8c06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:48.991324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f50588-303b-4c5f-8c8f-17aa98b8c06f admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:48.991608 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f50588-303b-4c5f-8c8f-17aa98b8c06f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:48.991884 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f50588-303b-4c5f-8c8f-17aa98b8c06f admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:48.992526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f50588-303b-4c5f-8c8f-17aa98b8c06f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:48.994470 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f50588-303b-4c5f-8c8f-17aa98b8c06f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:49.019093 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1327/2677] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:48 2022] POST /identity/v3/users => generated 303 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:49.020326 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295b524-55fa-47ce-9697-14d983febd27 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:49.030964 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295b524-55fa-47ce-9697-14d983febd27 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:49.035768 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15116e5-99c4-4001-b6ab-911f20db5f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:49.036363 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15116e5-99c4-4001-b6ab-911f20db5f80 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:49.036641 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15116e5-99c4-4001-b6ab-911f20db5f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:49.036912 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15116e5-99c4-4001-b6ab-911f20db5f80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:49.038122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15116e5-99c4-4001-b6ab-911f20db5f80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:49.039756 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15116e5-99c4-4001-b6ab-911f20db5f80 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:49.050461 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1351/2678] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:49.054313 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f50588-303b-4c5f-8c8f-17aa98b8c06f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:49.061124 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:49.064868 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f50588-303b-4c5f-8c8f-17aa98b8c06f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:49.071059 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f1ea2-6191-4ca3-af0b-cff9c1886227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:49.071780 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f1ea2-6191-4ca3-af0b-cff9c1886227 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:49.072063 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f1ea2-6191-4ca3-af0b-cff9c1886227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:49.072345 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f1ea2-6191-4ca3-af0b-cff9c1886227 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:49.073058 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f1ea2-6191-4ca3-af0b-cff9c1886227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:49.075291 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f1ea2-6191-4ca3-af0b-cff9c1886227 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:49.086842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1328/2679] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:49.090948 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15116e5-99c4-4001-b6ab-911f20db5f80 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:49.102174 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15116e5-99c4-4001-b6ab-911f20db5f80 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:49.107100 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2dc9a-ddb5-4be1-a501-dd44b5b1e3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:49.107789 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2dc9a-ddb5-4be1-a501-dd44b5b1e3c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:49.108071 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2dc9a-ddb5-4be1-a501-dd44b5b1e3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:49.108344 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2dc9a-ddb5-4be1-a501-dd44b5b1e3c9 admin admin] PATH_INFO: `/v3/projects/e01515254c414e698b8955fbb5e96147/users/2dc2be2e363b4907b49fb5678c6e0a49/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:49.124351 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2dc9a-ddb5-4be1-a501-dd44b5b1e3c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e01515254c414e698b8955fbb5e96147, user_id=2dc2be2e363b4907b49fb5678c6e0a49, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:49.127042 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2dc9a-ddb5-4be1-a501-dd44b5b1e3c9 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:49.141654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1352/2680] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:49 2022] PUT /identity/v3/projects/e01515254c414e698b8955fbb5e96147/users/2dc2be2e363b4907b49fb5678c6e0a49/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:49.147477 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1963b-2e01-4f44-b3e4-82aed3e4b26e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:49.147743 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1963b-2e01-4f44-b3e4-82aed3e4b26e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:49.148015 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1963b-2e01-4f44-b3e4-82aed3e4b26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:49.169374 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-35f1963b-2e01-4f44-b3e4-82aed3e4b26e None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:49.170734 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-35f1963b-2e01-4f44-b3e4-82aed3e4b26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:49.230175 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1329/2681] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:49.239676 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2dc9a-ddb5-4be1-a501-dd44b5b1e3c9 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:49.241545 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-66e2dc9a-ddb5-4be1-a501-dd44b5b1e3c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:49.298360 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2dc9a-ddb5-4be1-a501-dd44b5b1e3c9 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:49.301961 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364639c-dbe0-4fd1-bb95-1a74043650e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:49.302584 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364639c-dbe0-4fd1-bb95-1a74043650e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:49.302863 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364639c-dbe0-4fd1-bb95-1a74043650e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:49.303135 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364639c-dbe0-4fd1-bb95-1a74043650e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:49.312285 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364639c-dbe0-4fd1-bb95-1a74043650e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:49.313734 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364639c-dbe0-4fd1-bb95-1a74043650e8 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:49.323757 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1353/2682] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:49.952143 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ac011-5297-425c-86a7-4af281f10048 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:49.952662 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ac011-5297-425c-86a7-4af281f10048 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:49.952782 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ac011-5297-425c-86a7-4af281f10048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:49.981474 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7b6ac011-5297-425c-86a7-4af281f10048 None None] MFA Rules not processed for user `2dc2be2e363b4907b49fb5678c6e0a49`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:49.982733 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7b6ac011-5297-425c-86a7-4af281f10048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:50.050949 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1330/2683] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:43:49 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:50.055942 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364639c-dbe0-4fd1-bb95-1a74043650e8 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:50.066012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364639c-dbe0-4fd1-bb95-1a74043650e8 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:50.070783 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32296a-a596-4970-8fdc-22f0224fe2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:50.071403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32296a-a596-4970-8fdc-22f0224fe2f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:50.071683 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32296a-a596-4970-8fdc-22f0224fe2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:50.071957 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32296a-a596-4970-8fdc-22f0224fe2f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:50.072635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32296a-a596-4970-8fdc-22f0224fe2f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:50.074609 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32296a-a596-4970-8fdc-22f0224fe2f2 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:50.107748 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1354/2684] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:50 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:50.109358 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ac011-5297-425c-86a7-4af281f10048 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:50.121510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ac011-5297-425c-86a7-4af281f10048 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:50.187404 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92047c-7258-4e03-ab04-3eaf357baa77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:50.188071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92047c-7258-4e03-ab04-3eaf357baa77 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:50.188357 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92047c-7258-4e03-ab04-3eaf357baa77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:50.188663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92047c-7258-4e03-ab04-3eaf357baa77 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:50.189439 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92047c-7258-4e03-ab04-3eaf357baa77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:50.191584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92047c-7258-4e03-ab04-3eaf357baa77 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:50.214321 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1331/2685] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:50 2022] POST /identity/v3/users => generated 309 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:50.218449 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32296a-a596-4970-8fdc-22f0224fe2f2 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:50.229161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32296a-a596-4970-8fdc-22f0224fe2f2 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:50.233944 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd27aed-b670-4389-9f6b-e29daba375bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:50.234576 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd27aed-b670-4389-9f6b-e29daba375bc admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:50.234859 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd27aed-b670-4389-9f6b-e29daba375bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:50.235133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd27aed-b670-4389-9f6b-e29daba375bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:50.235781 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd27aed-b670-4389-9f6b-e29daba375bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:50.237378 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd27aed-b670-4389-9f6b-e29daba375bc admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:50.248943 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1355/2686] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:50.252980 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92047c-7258-4e03-ab04-3eaf357baa77 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:50.263083 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92047c-7258-4e03-ab04-3eaf357baa77 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:50.267915 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35384062-1901-447c-ab90-feb0c358bfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:50.268601 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35384062-1901-447c-ab90-feb0c358bfe9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:50.268889 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35384062-1901-447c-ab90-feb0c358bfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:50.269169 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35384062-1901-447c-ab90-feb0c358bfe9 admin admin] PATH_INFO: `/v3/projects/4329d8fe910f4b88849f286ed106f6e6/users/c72e568225184f6ba2105661f7e2b07e/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:50.281045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:50.286060 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35384062-1901-447c-ab90-feb0c358bfe9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4329d8fe910f4b88849f286ed106f6e6, user_id=c72e568225184f6ba2105661f7e2b07e, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:50.290041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35384062-1901-447c-ab90-feb0c358bfe9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:50.302884 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1332/2687] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:50 2022] PUT /identity/v3/projects/4329d8fe910f4b88849f286ed106f6e6/users/c72e568225184f6ba2105661f7e2b07e/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:50.306837 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd27aed-b670-4389-9f6b-e29daba375bc admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:50.316681 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd27aed-b670-4389-9f6b-e29daba375bc admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:50.372362 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5804187-eb8c-48ac-88f3-0fd706b4025e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:50.372965 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5804187-eb8c-48ac-88f3-0fd706b4025e admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:50.373496 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5804187-eb8c-48ac-88f3-0fd706b4025e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:50.373886 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5804187-eb8c-48ac-88f3-0fd706b4025e admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:50.374578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5804187-eb8c-48ac-88f3-0fd706b4025e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:50.376156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5804187-eb8c-48ac-88f3-0fd706b4025e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:50.385855 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1356/2688] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:50.389974 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35384062-1901-447c-ab90-feb0c358bfe9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:50.400724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35384062-1901-447c-ab90-feb0c358bfe9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:50.405408 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2a01f-b534-4737-92de-3c17f051a7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:50.406051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2a01f-b534-4737-92de-3c17f051a7a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:50.406373 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2a01f-b534-4737-92de-3c17f051a7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:50.406649 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2a01f-b534-4737-92de-3c17f051a7a0 admin admin] PATH_INFO: `/v3/projects/4329d8fe910f4b88849f286ed106f6e6/users/c72e568225184f6ba2105661f7e2b07e/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:50.408597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e2a01f-b534-4737-92de-3c17f051a7a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4329d8fe910f4b88849f286ed106f6e6, user_id=c72e568225184f6ba2105661f7e2b07e, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:50.411528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e2a01f-b534-4737-92de-3c17f051a7a0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:50.422109 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1333/2689] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:50 2022] PUT /identity/v3/projects/4329d8fe910f4b88849f286ed106f6e6/users/c72e568225184f6ba2105661f7e2b07e/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:51.119740 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff292e4-c734-40d7-9548-88cf5392e5fb None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:51.120283 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff292e4-c734-40d7-9548-88cf5392e5fb None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:51.120564 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff292e4-c734-40d7-9548-88cf5392e5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:51.129186 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:51.152431 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-6ff292e4-c734-40d7-9548-88cf5392e5fb None None] MFA Rules not processed for user `c72e568225184f6ba2105661f7e2b07e`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:51.153928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6ff292e4-c734-40d7-9548-88cf5392e5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:51.228035 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1357/2690] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:43:51 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:51.689080 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2a01f-b534-4737-92de-3c17f051a7a0 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:51.690786 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-36e2a01f-b534-4737-92de-3c17f051a7a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:51.778735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2a01f-b534-4737-92de-3c17f051a7a0 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:51.783986 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15696917-b970-4c7e-8557-f30362caa1d5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12302cdb2854b0684179647a6dadf6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:51.784856 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15696917-b970-4c7e-8557-f30362caa1d5 service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:51.785133 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15696917-b970-4c7e-8557-f30362caa1d5 service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:51.785403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15696917-b970-4c7e-8557-f30362caa1d5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:51.787158 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-15696917-b970-4c7e-8557-f30362caa1d5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:51.798838 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15696917-b970-4c7e-8557-f30362caa1d5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:51.800433 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15696917-b970-4c7e-8557-f30362caa1d5 service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:51.811015 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1334/2691] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:52.597120 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff292e4-c734-40d7-9548-88cf5392e5fb None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:52.663933 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff292e4-c734-40d7-9548-88cf5392e5fb None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:52.670697 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a22cc-2ff0-4548-9f32-83823c2ec238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:52.671324 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a22cc-2ff0-4548-9f32-83823c2ec238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:52.671607 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a22cc-2ff0-4548-9f32-83823c2ec238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:52.671882 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a22cc-2ff0-4548-9f32-83823c2ec238 admin admin] PATH_INFO: `/v3/users/2dc2be2e363b4907b49fb5678c6e0a49` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:52.673032 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a22cc-2ff0-4548-9f32-83823c2ec238 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dc2be2e363b4907b49fb5678c6e0a49)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:52.675186 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a22cc-2ff0-4548-9f32-83823c2ec238 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:52.721810 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ba4a22cc-2ff0-4548-9f32-83823c2ec238 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dc2be2e363b4907b49fb5678c6e0a49'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:52.729487 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ba4a22cc-2ff0-4548-9f32-83823c2ec238 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dc2be2e363b4907b49fb5678c6e0a49'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:52.730155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ba4a22cc-2ff0-4548-9f32-83823c2ec238 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dc2be2e363b4907b49fb5678c6e0a49'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:52.730799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ba4a22cc-2ff0-4548-9f32-83823c2ec238 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dc2be2e363b4907b49fb5678c6e0a49'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:52.742576 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ba4a22cc-2ff0-4548-9f32-83823c2ec238 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dc2be2e363b4907b49fb5678c6e0a49'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:52.756208 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1358/2692] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:52 2022] DELETE /identity/v3/users/2dc2be2e363b4907b49fb5678c6e0a49 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:52.944704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15696917-b970-4c7e-8557-f30362caa1d5 service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:52.956711 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15696917-b970-4c7e-8557-f30362caa1d5 service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:53.019421 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477dd5d4-75a2-4ad8-ac6f-f08cefa4756d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:53.020058 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477dd5d4-75a2-4ad8-ac6f-f08cefa4756d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:53.020339 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477dd5d4-75a2-4ad8-ac6f-f08cefa4756d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:53.020631 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477dd5d4-75a2-4ad8-ac6f-f08cefa4756d admin admin] PATH_INFO: `/v3/projects/e01515254c414e698b8955fbb5e96147` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:53.021769 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477dd5d4-75a2-4ad8-ac6f-f08cefa4756d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e01515254c414e698b8955fbb5e96147)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:53.024298 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477dd5d4-75a2-4ad8-ac6f-f08cefa4756d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:53.078962 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-477dd5d4-75a2-4ad8-ac6f-f08cefa4756d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e01515254c414e698b8955fbb5e96147', 'request_id': 'req-477dd5d4-75a2-4ad8-ac6f-f08cefa4756d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:53.085322 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-477dd5d4-75a2-4ad8-ac6f-f08cefa4756d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e01515254c414e698b8955fbb5e96147', 'request_id': 'req-477dd5d4-75a2-4ad8-ac6f-f08cefa4756d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:53.093120 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-477dd5d4-75a2-4ad8-ac6f-f08cefa4756d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e01515254c414e698b8955fbb5e96147', 'request_id': 'req-477dd5d4-75a2-4ad8-ac6f-f08cefa4756d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:53.097842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:53.106468 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-477dd5d4-75a2-4ad8-ac6f-f08cefa4756d admin admin] The token cache is being invalidate because project e01515254c414e698b8955fbb5e96147 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:53.106951 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-477dd5d4-75a2-4ad8-ac6f-f08cefa4756d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:53.107864 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-477dd5d4-75a2-4ad8-ac6f-f08cefa4756d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:53.109399 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1335/2693] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:52 2022] DELETE /identity/v3/projects/e01515254c414e698b8955fbb5e96147 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:53.113475 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a22cc-2ff0-4548-9f32-83823c2ec238 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:53.114981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ba4a22cc-2ff0-4548-9f32-83823c2ec238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:53.128875 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a22cc-2ff0-4548-9f32-83823c2ec238 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:53.133717 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:53.199825 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624ab8b5-9384-4e50-b6ea-502fc63810b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:53.200519 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ab8b5-9384-4e50-b6ea-502fc63810b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:53.200951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ab8b5-9384-4e50-b6ea-502fc63810b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:53.201227 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ab8b5-9384-4e50-b6ea-502fc63810b2 admin admin] PATH_INFO: `/v3/users/c72e568225184f6ba2105661f7e2b07e` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:53.202511 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624ab8b5-9384-4e50-b6ea-502fc63810b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c72e568225184f6ba2105661f7e2b07e)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:53.204766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624ab8b5-9384-4e50-b6ea-502fc63810b2 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:53.253835 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-624ab8b5-9384-4e50-b6ea-502fc63810b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c72e568225184f6ba2105661f7e2b07e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:53.261025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-624ab8b5-9384-4e50-b6ea-502fc63810b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c72e568225184f6ba2105661f7e2b07e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:53.261805 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-624ab8b5-9384-4e50-b6ea-502fc63810b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c72e568225184f6ba2105661f7e2b07e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:53.262459 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-624ab8b5-9384-4e50-b6ea-502fc63810b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c72e568225184f6ba2105661f7e2b07e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:53.275415 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-624ab8b5-9384-4e50-b6ea-502fc63810b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c72e568225184f6ba2105661f7e2b07e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:53.291024 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1359/2694] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:53 2022] DELETE /identity/v3/users/c72e568225184f6ba2105661f7e2b07e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:53.470611 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477dd5d4-75a2-4ad8-ac6f-f08cefa4756d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:53.472126 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-477dd5d4-75a2-4ad8-ac6f-f08cefa4756d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:53.486907 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477dd5d4-75a2-4ad8-ac6f-f08cefa4756d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:53.546150 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe5452-133f-4f2f-b819-ac65a8a59a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:53.546743 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe5452-133f-4f2f-b819-ac65a8a59a48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:53.547036 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe5452-133f-4f2f-b819-ac65a8a59a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:53.547315 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe5452-133f-4f2f-b819-ac65a8a59a48 admin admin] PATH_INFO: `/v3/projects/4329d8fe910f4b88849f286ed106f6e6` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:53.548428 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe5452-133f-4f2f-b819-ac65a8a59a48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4329d8fe910f4b88849f286ed106f6e6)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:53.550367 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe5452-133f-4f2f-b819-ac65a8a59a48 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:53.596623 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1cfe5452-133f-4f2f-b819-ac65a8a59a48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4329d8fe910f4b88849f286ed106f6e6', 'request_id': 'req-1cfe5452-133f-4f2f-b819-ac65a8a59a48'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:53.603431 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1cfe5452-133f-4f2f-b819-ac65a8a59a48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4329d8fe910f4b88849f286ed106f6e6', 'request_id': 'req-1cfe5452-133f-4f2f-b819-ac65a8a59a48'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:53.610362 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1cfe5452-133f-4f2f-b819-ac65a8a59a48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4329d8fe910f4b88849f286ed106f6e6', 'request_id': 'req-1cfe5452-133f-4f2f-b819-ac65a8a59a48'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:53.620825 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1cfe5452-133f-4f2f-b819-ac65a8a59a48 admin admin] The token cache is being invalidate because project 4329d8fe910f4b88849f286ed106f6e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:53.621115 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1cfe5452-133f-4f2f-b819-ac65a8a59a48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:53.621731 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1cfe5452-133f-4f2f-b819-ac65a8a59a48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:53.623231 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1336/2695] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:53 2022] DELETE /identity/v3/projects/4329d8fe910f4b88849f286ed106f6e6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:53.629399 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5275ee-26f0-466d-a389-147d35cc8fb9 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:53.629910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5275ee-26f0-466d-a389-147d35cc8fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:53.630013 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5275ee-26f0-466d-a389-147d35cc8fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:53.650503 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-8a5275ee-26f0-466d-a389-147d35cc8fb9 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:53.651701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8a5275ee-26f0-466d-a389-147d35cc8fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:53.713210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1360/2696] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:53.720837 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47fb9e-ed05-4309-b41d-22c9f6e48439 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:53.721122 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47fb9e-ed05-4309-b41d-22c9f6e48439 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:53.721393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47fb9e-ed05-4309-b41d-22c9f6e48439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:53.741299 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fe47fb9e-ed05-4309-b41d-22c9f6e48439 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:53.742545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fe47fb9e-ed05-4309-b41d-22c9f6e48439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:53.750485 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1337/2697] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:53.754506 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5275ee-26f0-466d-a389-147d35cc8fb9 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:53.764626 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5275ee-26f0-466d-a389-147d35cc8fb9 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:53.769300 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03d0ac-52e2-477d-b4d6-9c4f84d6b0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:53.769894 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03d0ac-52e2-477d-b4d6-9c4f84d6b0ac admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:53.770178 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03d0ac-52e2-477d-b4d6-9c4f84d6b0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:53.770483 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03d0ac-52e2-477d-b4d6-9c4f84d6b0ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:53.771133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03d0ac-52e2-477d-b4d6-9c4f84d6b0ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:43:53.771439 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03d0ac-52e2-477d-b4d6-9c4f84d6b0ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:53.773816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03d0ac-52e2-477d-b4d6-9c4f84d6b0ac admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:53.785561 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1361/2698] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:43:53.789681 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47fb9e-ed05-4309-b41d-22c9f6e48439 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:53.799040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47fb9e-ed05-4309-b41d-22c9f6e48439 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:53.803569 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b83c3-df42-48d7-8753-af5add0cff98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:53.804271 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b83c3-df42-48d7-8753-af5add0cff98 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:53.804554 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b83c3-df42-48d7-8753-af5add0cff98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:53.804830 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b83c3-df42-48d7-8753-af5add0cff98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:53.805471 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b83c3-df42-48d7-8753-af5add0cff98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:53.807467 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b83c3-df42-48d7-8753-af5add0cff98 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:53.833957 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1338/2699] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:53 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:53.838623 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03d0ac-52e2-477d-b4d6-9c4f84d6b0ac admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:53.848616 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03d0ac-52e2-477d-b4d6-9c4f84d6b0ac admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:53.859952 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:53.910769 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4fb38-61bd-441e-921b-acff01b2514f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:53.911366 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4fb38-61bd-441e-921b-acff01b2514f admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:53.911647 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4fb38-61bd-441e-921b-acff01b2514f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:53.911916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4fb38-61bd-441e-921b-acff01b2514f admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:53.912547 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d4fb38-61bd-441e-921b-acff01b2514f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:53.914470 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d4fb38-61bd-441e-921b-acff01b2514f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:53.935657 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1362/2700] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:53 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:53.939918 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b83c3-df42-48d7-8753-af5add0cff98 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:53.950699 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b83c3-df42-48d7-8753-af5add0cff98 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:53.955305 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a34e8-f05e-4fbd-81c1-a808067b6321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:53.955902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a34e8-f05e-4fbd-81c1-a808067b6321 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:53.956185 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a34e8-f05e-4fbd-81c1-a808067b6321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:53.956459 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a34e8-f05e-4fbd-81c1-a808067b6321 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:53.957107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a34e8-f05e-4fbd-81c1-a808067b6321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:53.958808 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a34e8-f05e-4fbd-81c1-a808067b6321 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:53.969293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1339/2701] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:53.973161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4fb38-61bd-441e-921b-acff01b2514f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:53.983352 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4fb38-61bd-441e-921b-acff01b2514f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:53.988375 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c3a532-0890-464b-b17a-c4bf7fc80934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:53.988976 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3a532-0890-464b-b17a-c4bf7fc80934 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:53.989287 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3a532-0890-464b-b17a-c4bf7fc80934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:53.989564 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3a532-0890-464b-b17a-c4bf7fc80934 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:53.990243 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c3a532-0890-464b-b17a-c4bf7fc80934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:53.991841 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c3a532-0890-464b-b17a-c4bf7fc80934 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:54.002630 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1363/2702] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:54.007286 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a34e8-f05e-4fbd-81c1-a808067b6321 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:54.017389 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a34e8-f05e-4fbd-81c1-a808067b6321 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:54.022813 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1865632-a363-427e-9a2d-182c37c9d279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:54.023431 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1865632-a363-427e-9a2d-182c37c9d279 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:54.023712 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1865632-a363-427e-9a2d-182c37c9d279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:54.023985 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1865632-a363-427e-9a2d-182c37c9d279 admin admin] PATH_INFO: `/v3/projects/6e6c771ecb6d46e4801243bab698c5ef/users/24a96b3280a245d09d23d405eba3084a/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:54.038581 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1865632-a363-427e-9a2d-182c37c9d279 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e6c771ecb6d46e4801243bab698c5ef, user_id=24a96b3280a245d09d23d405eba3084a, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:54.041386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1865632-a363-427e-9a2d-182c37c9d279 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:54.055684 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1340/2703] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:54 2022] PUT /identity/v3/projects/6e6c771ecb6d46e4801243bab698c5ef/users/24a96b3280a245d09d23d405eba3084a/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:54.067077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96393b5-9bbd-44e5-8228-4134cc4b7d00 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:54.067362 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96393b5-9bbd-44e5-8228-4134cc4b7d00 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:54.067637 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96393b5-9bbd-44e5-8228-4134cc4b7d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:54.096372 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e96393b5-9bbd-44e5-8228-4134cc4b7d00 None None] MFA Rules not processed for user `24a96b3280a245d09d23d405eba3084a`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:54.097583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e96393b5-9bbd-44e5-8228-4134cc4b7d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:54.165565 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1364/2704] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:43:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:54.170115 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1865632-a363-427e-9a2d-182c37c9d279 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:54.195965 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:54.234846 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1865632-a363-427e-9a2d-182c37c9d279 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:54.239597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3fea2f-90a9-45a1-845e-d3b5fb1d77b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:54.240195 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fea2f-90a9-45a1-845e-d3b5fb1d77b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:54.240477 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fea2f-90a9-45a1-845e-d3b5fb1d77b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:54.241286 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fea2f-90a9-45a1-845e-d3b5fb1d77b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:54.241942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3fea2f-90a9-45a1-845e-d3b5fb1d77b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:54.243903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3fea2f-90a9-45a1-845e-d3b5fb1d77b8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:54.271231 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1341/2705] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:54 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:54.275374 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96393b5-9bbd-44e5-8228-4134cc4b7d00 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:54.286439 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96393b5-9bbd-44e5-8228-4134cc4b7d00 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:54.341928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1dae09-5c2c-4db9-a763-46a5938426b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:54.342488 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1dae09-5c2c-4db9-a763-46a5938426b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:54.342900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1dae09-5c2c-4db9-a763-46a5938426b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:54.343181 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1dae09-5c2c-4db9-a763-46a5938426b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:54.343821 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1dae09-5c2c-4db9-a763-46a5938426b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:54.345749 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1dae09-5c2c-4db9-a763-46a5938426b5 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:54.369842 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1365/2706] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:54 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:54.374330 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3fea2f-90a9-45a1-845e-d3b5fb1d77b8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:54.385868 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3fea2f-90a9-45a1-845e-d3b5fb1d77b8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:54.390585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ef9a8-ca68-49ca-9810-1a0fe8ae9336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:54.391072 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ef9a8-ca68-49ca-9810-1a0fe8ae9336 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:54.391354 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ef9a8-ca68-49ca-9810-1a0fe8ae9336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:54.391664 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ef9a8-ca68-49ca-9810-1a0fe8ae9336 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:54.392258 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ef9a8-ca68-49ca-9810-1a0fe8ae9336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:54.393830 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ef9a8-ca68-49ca-9810-1a0fe8ae9336 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:54.404918 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1342/2707] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:54.408939 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1dae09-5c2c-4db9-a763-46a5938426b5 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:54.419856 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1dae09-5c2c-4db9-a763-46a5938426b5 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:54.424672 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0fb33-0a49-4e5d-8132-94cf89f964bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:54.425329 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0fb33-0a49-4e5d-8132-94cf89f964bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:54.425619 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0fb33-0a49-4e5d-8132-94cf89f964bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:54.425898 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0fb33-0a49-4e5d-8132-94cf89f964bb admin admin] PATH_INFO: `/v3/projects/2fefb41752a6453881493309d47062c6/users/7137b82b2fba432ca9918904d78650c9/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:54.439772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:54.444379 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0fb33-0a49-4e5d-8132-94cf89f964bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fefb41752a6453881493309d47062c6, user_id=7137b82b2fba432ca9918904d78650c9, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:54.447562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0fb33-0a49-4e5d-8132-94cf89f964bb admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:54.463129 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1366/2708] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:54 2022] PUT /identity/v3/projects/2fefb41752a6453881493309d47062c6/users/7137b82b2fba432ca9918904d78650c9/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:54.467080 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ef9a8-ca68-49ca-9810-1a0fe8ae9336 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:54.478165 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ef9a8-ca68-49ca-9810-1a0fe8ae9336 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:54.536009 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68f9b8-2dbc-4f53-b165-a9ae87e648f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:54.536748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68f9b8-2dbc-4f53-b165-a9ae87e648f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:54.537039 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68f9b8-2dbc-4f53-b165-a9ae87e648f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:54.537319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68f9b8-2dbc-4f53-b165-a9ae87e648f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:54.538588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68f9b8-2dbc-4f53-b165-a9ae87e648f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:54.540181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68f9b8-2dbc-4f53-b165-a9ae87e648f8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:54.550464 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1343/2709] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:43:54.555059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0fb33-0a49-4e5d-8132-94cf89f964bb admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:54.565757 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0fb33-0a49-4e5d-8132-94cf89f964bb admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:54.570994 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d7d6f0-ddae-428d-a80d-469d11c6dd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:54.571902 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7d6f0-ddae-428d-a80d-469d11c6dd2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:54.572235 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7d6f0-ddae-428d-a80d-469d11c6dd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:54.572518 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7d6f0-ddae-428d-a80d-469d11c6dd2c admin admin] PATH_INFO: `/v3/projects/2fefb41752a6453881493309d47062c6/users/7137b82b2fba432ca9918904d78650c9/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:54.575248 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7d6f0-ddae-428d-a80d-469d11c6dd2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fefb41752a6453881493309d47062c6, user_id=7137b82b2fba432ca9918904d78650c9, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:54.578254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7d6f0-ddae-428d-a80d-469d11c6dd2c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:54.588676 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1367/2710] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:54 2022] PUT /identity/v3/projects/2fefb41752a6453881493309d47062c6/users/7137b82b2fba432ca9918904d78650c9/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:54.599019 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd79736-914f-4349-a839-8814fa37c9e1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:54.599361 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd79736-914f-4349-a839-8814fa37c9e1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:54.599645 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd79736-914f-4349-a839-8814fa37c9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:54.628504 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-bbd79736-914f-4349-a839-8814fa37c9e1 None None] MFA Rules not processed for user `7137b82b2fba432ca9918904d78650c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:54.629783 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bbd79736-914f-4349-a839-8814fa37c9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:54.697768 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1344/2711] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:43:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:54.708961 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d7d6f0-ddae-428d-a80d-469d11c6dd2c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:54.710734 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f9d7d6f0-ddae-428d-a80d-469d11c6dd2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:54.768520 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d7d6f0-ddae-428d-a80d-469d11c6dd2c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:54.772624 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8936a56-778f-4d83-bfbc-2f7b532cf9bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:54.773213 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8936a56-778f-4d83-bfbc-2f7b532cf9bd service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:54.773488 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8936a56-778f-4d83-bfbc-2f7b532cf9bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:54.773759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8936a56-778f-4d83-bfbc-2f7b532cf9bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:54.782405 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8936a56-778f-4d83-bfbc-2f7b532cf9bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:54.783814 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8936a56-778f-4d83-bfbc-2f7b532cf9bd service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:54.795093 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1368/2712] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:54 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:54.869022 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd79736-914f-4349-a839-8814fa37c9e1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:54.879228 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd79736-914f-4349-a839-8814fa37c9e1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:54.882856 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d74a5-451f-4ba5-8692-f5905bc546b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:54.883452 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d74a5-451f-4ba5-8692-f5905bc546b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:54.883729 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d74a5-451f-4ba5-8692-f5905bc546b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:54.883999 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d74a5-451f-4ba5-8692-f5905bc546b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:54.896717 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:54.944509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d74a5-451f-4ba5-8692-f5905bc546b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:54.945939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d74a5-451f-4ba5-8692-f5905bc546b0 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:54.954927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1345/2713] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:56.766969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fac2b-003d-4e9f-82b6-81e74b60e41e None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:56.767423 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fac2b-003d-4e9f-82b6-81e74b60e41e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:56.767521 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fac2b-003d-4e9f-82b6-81e74b60e41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:56.786302 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-190fac2b-003d-4e9f-82b6-81e74b60e41e None None] MFA Rules not processed for user `3fbb843ada72400e9d9f06521c2a61ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:56.787498 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-190fac2b-003d-4e9f-82b6-81e74b60e41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:56.841020 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1369/2714] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:43:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:04.950755 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=20 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:06.834580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=60,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:31.039606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e5448-95da-41bf-b748-ae7afa11c7b6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:31.040119 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e5448-95da-41bf-b748-ae7afa11c7b6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:31.040249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e5448-95da-41bf-b748-ae7afa11c7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:31.060091 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-967e5448-95da-41bf-b748-ae7afa11c7b6 None None] MFA Rules not processed for user `dd7b98536fa445cebd552fcbf54b757d`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:31.061311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-967e5448-95da-41bf-b748-ae7afa11c7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:31.114998 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1346/2715] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Sep 1 16:44:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:31.166667 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fac2b-003d-4e9f-82b6-81e74b60e41e None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:31.168173 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-190fac2b-003d-4e9f-82b6-81e74b60e41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:31.249517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fac2b-003d-4e9f-82b6-81e74b60e41e None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:31.256014 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04158d5d-a36c-4daa-981e-12ff7f1c7f69 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e789695517c1437fb224f5a6195655f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:31.256737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04158d5d-a36c-4daa-981e-12ff7f1c7f69 service swift] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:31.257025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04158d5d-a36c-4daa-981e-12ff7f1c7f69 service swift] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:31.257302 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04158d5d-a36c-4daa-981e-12ff7f1c7f69 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:31.266583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04158d5d-a36c-4daa-981e-12ff7f1c7f69 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:31.268235 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04158d5d-a36c-4daa-981e-12ff7f1c7f69 service swift] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:31.277577 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1370/2716] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Sep 1 16:44:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:44:41.108315 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=25 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:41.273383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=37 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:46.325895 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967e5448-95da-41bf-b748-ae7afa11c7b6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:46.392559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967e5448-95da-41bf-b748-ae7afa11c7b6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:46.397045 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b161520-cfd8-4b5d-b34c-bceb7b86a9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:46.397671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b161520-cfd8-4b5d-b34c-bceb7b86a9a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:46.397959 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b161520-cfd8-4b5d-b34c-bceb7b86a9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:46.398265 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b161520-cfd8-4b5d-b34c-bceb7b86a9a7 admin admin] PATH_INFO: `/v3/users/24a96b3280a245d09d23d405eba3084a` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:46.399387 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b161520-cfd8-4b5d-b34c-bceb7b86a9a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24a96b3280a245d09d23d405eba3084a)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:46.401296 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b161520-cfd8-4b5d-b34c-bceb7b86a9a7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:46.470375 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5b161520-cfd8-4b5d-b34c-bceb7b86a9a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24a96b3280a245d09d23d405eba3084a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:46.478035 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5b161520-cfd8-4b5d-b34c-bceb7b86a9a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24a96b3280a245d09d23d405eba3084a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:46.478777 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5b161520-cfd8-4b5d-b34c-bceb7b86a9a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24a96b3280a245d09d23d405eba3084a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:46.489388 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5b161520-cfd8-4b5d-b34c-bceb7b86a9a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24a96b3280a245d09d23d405eba3084a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:46.503702 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5b161520-cfd8-4b5d-b34c-bceb7b86a9a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24a96b3280a245d09d23d405eba3084a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:46.509897 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1347/2717] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:44:46 2022] DELETE /identity/v3/users/24a96b3280a245d09d23d405eba3084a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:46.516688 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0903227-9859-4ce0-85d1-92f815484ce6 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:46.517332 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0903227-9859-4ce0-85d1-92f815484ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:46.517332 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0903227-9859-4ce0-85d1-92f815484ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:46.538022 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d0903227-9859-4ce0-85d1-92f815484ce6 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:46.539525 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d0903227-9859-4ce0-85d1-92f815484ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:46.596966 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1371/2718] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:44:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:46.608774 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b161520-cfd8-4b5d-b34c-bceb7b86a9a7 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:46.610171 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5b161520-cfd8-4b5d-b34c-bceb7b86a9a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:46.668583 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b161520-cfd8-4b5d-b34c-bceb7b86a9a7 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:46.672153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b63300-10b6-4b7e-9431-41ae8fc3397a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:46.672759 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b63300-10b6-4b7e-9431-41ae8fc3397a service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:46.673040 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b63300-10b6-4b7e-9431-41ae8fc3397a service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:46.673312 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b63300-10b6-4b7e-9431-41ae8fc3397a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:46.682032 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b63300-10b6-4b7e-9431-41ae8fc3397a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:46.684010 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b63300-10b6-4b7e-9431-41ae8fc3397a service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:46.693396 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1348/2719] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:44:46.962091 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0903227-9859-4ce0-85d1-92f815484ce6 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:46.963674 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d0903227-9859-4ce0-85d1-92f815484ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:46.976547 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0903227-9859-4ce0-85d1-92f815484ce6 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:46.981650 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4405511-f3f8-4b9e-9e99-61503cb877ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:46.982307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4405511-f3f8-4b9e-9e99-61503cb877ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:46.982591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4405511-f3f8-4b9e-9e99-61503cb877ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:46.982864 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4405511-f3f8-4b9e-9e99-61503cb877ee admin admin] PATH_INFO: `/v3/projects/6e6c771ecb6d46e4801243bab698c5ef` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:46.983987 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4405511-f3f8-4b9e-9e99-61503cb877ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e6c771ecb6d46e4801243bab698c5ef)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:46.986056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4405511-f3f8-4b9e-9e99-61503cb877ee admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:47.033814 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b4405511-f3f8-4b9e-9e99-61503cb877ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e6c771ecb6d46e4801243bab698c5ef', 'request_id': 'req-b4405511-f3f8-4b9e-9e99-61503cb877ee'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:47.040281 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b4405511-f3f8-4b9e-9e99-61503cb877ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e6c771ecb6d46e4801243bab698c5ef', 'request_id': 'req-b4405511-f3f8-4b9e-9e99-61503cb877ee'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:47.046476 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b4405511-f3f8-4b9e-9e99-61503cb877ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e6c771ecb6d46e4801243bab698c5ef', 'request_id': 'req-b4405511-f3f8-4b9e-9e99-61503cb877ee'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:47.060094 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b4405511-f3f8-4b9e-9e99-61503cb877ee admin admin] The token cache is being invalidate because project 6e6c771ecb6d46e4801243bab698c5ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:44:47.060414 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b4405511-f3f8-4b9e-9e99-61503cb877ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:47.061053 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b4405511-f3f8-4b9e-9e99-61503cb877ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:47.062613 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1372/2720] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:44:46 2022] DELETE /identity/v3/projects/6e6c771ecb6d46e4801243bab698c5ef => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:47.066630 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b63300-10b6-4b7e-9431-41ae8fc3397a service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:47.076353 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b63300-10b6-4b7e-9431-41ae8fc3397a service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:47.094965 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:47.136372 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975d1a9-6b60-4dd6-bd16-805b167e6908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:47.137005 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975d1a9-6b60-4dd6-bd16-805b167e6908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:47.137288 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975d1a9-6b60-4dd6-bd16-805b167e6908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:47.137567 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975d1a9-6b60-4dd6-bd16-805b167e6908 admin admin] PATH_INFO: `/v3/users/7137b82b2fba432ca9918904d78650c9` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:47.138792 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8975d1a9-6b60-4dd6-bd16-805b167e6908 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7137b82b2fba432ca9918904d78650c9)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:47.140872 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8975d1a9-6b60-4dd6-bd16-805b167e6908 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:47.188371 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cebdba-8828-4a64-8785-9650acf0d72b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:47.188665 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cebdba-8828-4a64-8785-9650acf0d72b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:47.188947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cebdba-8828-4a64-8785-9650acf0d72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:47.192796 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8975d1a9-6b60-4dd6-bd16-805b167e6908 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7137b82b2fba432ca9918904d78650c9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:47.200247 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8975d1a9-6b60-4dd6-bd16-805b167e6908 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7137b82b2fba432ca9918904d78650c9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:47.200927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8975d1a9-6b60-4dd6-bd16-805b167e6908 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7137b82b2fba432ca9918904d78650c9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:47.208463 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8975d1a9-6b60-4dd6-bd16-805b167e6908 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7137b82b2fba432ca9918904d78650c9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:47.208756 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-24cebdba-8828-4a64-8785-9650acf0d72b None None] MFA Rules not processed for user `dd7b98536fa445cebd552fcbf54b757d`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:47.209978 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-24cebdba-8828-4a64-8785-9650acf0d72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:47.221202 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8975d1a9-6b60-4dd6-bd16-805b167e6908 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7137b82b2fba432ca9918904d78650c9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:47.227392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1349/2721] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:44:47 2022] DELETE /identity/v3/users/7137b82b2fba432ca9918904d78650c9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:44:47.269089 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1373/2722] 10.222.0.58 () {58 vars in 967 bytes} [Thu Sep 1 16:44:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:44:47.282905 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975d1a9-6b60-4dd6-bd16-805b167e6908 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:47.284419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8975d1a9-6b60-4dd6-bd16-805b167e6908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:47.358519 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975d1a9-6b60-4dd6-bd16-805b167e6908 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:47.363675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea78e8-0ce9-49d7-bc0d-8f7614cbb51d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e789695517c1437fb224f5a6195655f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:47.364293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea78e8-0ce9-49d7-bc0d-8f7614cbb51d service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:47.364600 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea78e8-0ce9-49d7-bc0d-8f7614cbb51d service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:47.364862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea78e8-0ce9-49d7-bc0d-8f7614cbb51d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:47.411377 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:47.427960 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea78e8-0ce9-49d7-bc0d-8f7614cbb51d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:47.429558 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea78e8-0ce9-49d7-bc0d-8f7614cbb51d service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:47.438427 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1350/2723] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Sep 1 16:44:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:44:47.503267 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cebdba-8828-4a64-8785-9650acf0d72b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:47.504780 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-24cebdba-8828-4a64-8785-9650acf0d72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:47.521005 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cebdba-8828-4a64-8785-9650acf0d72b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:47.552487 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:47.587830 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413881f1-bccc-48f7-a222-b708ad579021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:47.590635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413881f1-bccc-48f7-a222-b708ad579021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:47.590921 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413881f1-bccc-48f7-a222-b708ad579021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:47.591279 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413881f1-bccc-48f7-a222-b708ad579021 admin admin] PATH_INFO: `/v3/projects/2fefb41752a6453881493309d47062c6` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:47.592364 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413881f1-bccc-48f7-a222-b708ad579021 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fefb41752a6453881493309d47062c6)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:47.594319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413881f1-bccc-48f7-a222-b708ad579021 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:47.657023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-413881f1-bccc-48f7-a222-b708ad579021 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fefb41752a6453881493309d47062c6', 'request_id': 'req-413881f1-bccc-48f7-a222-b708ad579021'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:47.666527 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-413881f1-bccc-48f7-a222-b708ad579021 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fefb41752a6453881493309d47062c6', 'request_id': 'req-413881f1-bccc-48f7-a222-b708ad579021'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:47.672813 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-413881f1-bccc-48f7-a222-b708ad579021 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fefb41752a6453881493309d47062c6', 'request_id': 'req-413881f1-bccc-48f7-a222-b708ad579021'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:47.685597 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-413881f1-bccc-48f7-a222-b708ad579021 admin admin] The token cache is being invalidate because project 2fefb41752a6453881493309d47062c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:44:47.685890 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-413881f1-bccc-48f7-a222-b708ad579021 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:47.686559 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-413881f1-bccc-48f7-a222-b708ad579021 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:47.688044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1374/2724] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:44:47 2022] DELETE /identity/v3/projects/2fefb41752a6453881493309d47062c6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:47.693239 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f56ee-b025-40ed-bed1-569a13cbbd47 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:47.693517 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f56ee-b025-40ed-bed1-569a13cbbd47 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:47.694041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f56ee-b025-40ed-bed1-569a13cbbd47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:47.716378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-931f56ee-b025-40ed-bed1-569a13cbbd47 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:47.717602 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-931f56ee-b025-40ed-bed1-569a13cbbd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:47.780813 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1351/2725] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:44:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:47.788711 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b35370-f249-47d7-b8ed-15692d4bf555 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:47.789331 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b35370-f249-47d7-b8ed-15692d4bf555 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:47.789678 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b35370-f249-47d7-b8ed-15692d4bf555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:47.811157 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-75b35370-f249-47d7-b8ed-15692d4bf555 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:47.812474 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-75b35370-f249-47d7-b8ed-15692d4bf555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:47.820766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1375/2726] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:44:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:47.825791 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f56ee-b025-40ed-bed1-569a13cbbd47 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:47.837639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f56ee-b025-40ed-bed1-569a13cbbd47 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:47.844001 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779edef-ba06-43e9-b2da-98aa8097bfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:47.844607 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779edef-ba06-43e9-b2da-98aa8097bfd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:47.844889 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779edef-ba06-43e9-b2da-98aa8097bfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:47.845193 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779edef-ba06-43e9-b2da-98aa8097bfd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:47.845845 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779edef-ba06-43e9-b2da-98aa8097bfd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:44:47.846155 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779edef-ba06-43e9-b2da-98aa8097bfd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:47.847826 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779edef-ba06-43e9-b2da-98aa8097bfd7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:47.859941 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1352/2727] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:44:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:44:47.864503 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b35370-f249-47d7-b8ed-15692d4bf555 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:47.873662 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b35370-f249-47d7-b8ed-15692d4bf555 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:47.878243 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a830fe5-c521-417a-aa8a-87f0431c6a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:47.878833 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a830fe5-c521-417a-aa8a-87f0431c6a1e admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:47.879108 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a830fe5-c521-417a-aa8a-87f0431c6a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:47.879373 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a830fe5-c521-417a-aa8a-87f0431c6a1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:47.879995 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a830fe5-c521-417a-aa8a-87f0431c6a1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:47.881855 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a830fe5-c521-417a-aa8a-87f0431c6a1e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:47.910064 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1376/2728] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:44:47 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:44:47.914454 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779edef-ba06-43e9-b2da-98aa8097bfd7 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:47.925798 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779edef-ba06-43e9-b2da-98aa8097bfd7 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:47.983938 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1755fb-fdc1-4e7e-b114-34bc68099720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:47.984536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1755fb-fdc1-4e7e-b114-34bc68099720 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:47.984816 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1755fb-fdc1-4e7e-b114-34bc68099720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:47.985088 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1755fb-fdc1-4e7e-b114-34bc68099720 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:47.985712 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1755fb-fdc1-4e7e-b114-34bc68099720 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:47.987642 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1755fb-fdc1-4e7e-b114-34bc68099720 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:48.009145 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1353/2729] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:44:47 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:44:48.013806 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a830fe5-c521-417a-aa8a-87f0431c6a1e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:48.023916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a830fe5-c521-417a-aa8a-87f0431c6a1e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:48.028542 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5867302-12e2-426c-8d7a-a0c4a1ee88e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:48.029125 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5867302-12e2-426c-8d7a-a0c4a1ee88e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:48.029403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5867302-12e2-426c-8d7a-a0c4a1ee88e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:48.029678 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5867302-12e2-426c-8d7a-a0c4a1ee88e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:48.030359 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5867302-12e2-426c-8d7a-a0c4a1ee88e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:48.031936 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5867302-12e2-426c-8d7a-a0c4a1ee88e1 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:48.166850 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1377/2730] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:44:48.170961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1755fb-fdc1-4e7e-b114-34bc68099720 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:48.181028 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1755fb-fdc1-4e7e-b114-34bc68099720 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:48.186262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf82cf-38c9-48b7-bf84-46ab72bef735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:48.186855 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf82cf-38c9-48b7-bf84-46ab72bef735 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:48.187137 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf82cf-38c9-48b7-bf84-46ab72bef735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:48.187409 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf82cf-38c9-48b7-bf84-46ab72bef735 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:48.188053 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf82cf-38c9-48b7-bf84-46ab72bef735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:48.189641 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf82cf-38c9-48b7-bf84-46ab72bef735 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:48.201009 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1354/2731] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:44:48.204989 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5867302-12e2-426c-8d7a-a0c4a1ee88e1 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:48.214767 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5867302-12e2-426c-8d7a-a0c4a1ee88e1 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:48.219323 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22022387-7bcc-4f7f-b40f-e940264e0115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:48.219986 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22022387-7bcc-4f7f-b40f-e940264e0115 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:48.220268 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22022387-7bcc-4f7f-b40f-e940264e0115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:48.220542 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22022387-7bcc-4f7f-b40f-e940264e0115 admin admin] PATH_INFO: `/v3/projects/d17194312d83471f9c25716be39139ce/users/8404a4ada2084aceb0019d51855c71c0/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:48.234896 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22022387-7bcc-4f7f-b40f-e940264e0115 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d17194312d83471f9c25716be39139ce, user_id=8404a4ada2084aceb0019d51855c71c0, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:48.237518 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22022387-7bcc-4f7f-b40f-e940264e0115 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:48.251278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1378/2732] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:44:48 2022] PUT /identity/v3/projects/d17194312d83471f9c25716be39139ce/users/8404a4ada2084aceb0019d51855c71c0/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:48.262340 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08893630-8784-4e92-a211-1abdd60bd576 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:48.262629 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08893630-8784-4e92-a211-1abdd60bd576 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:48.262903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08893630-8784-4e92-a211-1abdd60bd576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:48.290790 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-08893630-8784-4e92-a211-1abdd60bd576 None None] MFA Rules not processed for user `8404a4ada2084aceb0019d51855c71c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:48.292029 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-08893630-8784-4e92-a211-1abdd60bd576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:48.330499 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:48.358599 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1355/2733] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:44:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:48.363138 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22022387-7bcc-4f7f-b40f-e940264e0115 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:48.422158 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:48.427962 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22022387-7bcc-4f7f-b40f-e940264e0115 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:48.434173 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50e5c5-5515-4df4-a584-3243f20909c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:48.434800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50e5c5-5515-4df4-a584-3243f20909c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:48.435074 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50e5c5-5515-4df4-a584-3243f20909c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:48.435340 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50e5c5-5515-4df4-a584-3243f20909c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:48.435969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c50e5c5-5515-4df4-a584-3243f20909c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:48.437864 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c50e5c5-5515-4df4-a584-3243f20909c1 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:48.463489 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1379/2734] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:44:48 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:44:48.467543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08893630-8784-4e92-a211-1abdd60bd576 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:48.477903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08893630-8784-4e92-a211-1abdd60bd576 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:48.533999 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72d4ec-0f9d-4cd3-b86c-b63f02962ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:48.534627 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72d4ec-0f9d-4cd3-b86c-b63f02962ea1 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:48.534907 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72d4ec-0f9d-4cd3-b86c-b63f02962ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:48.535179 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72d4ec-0f9d-4cd3-b86c-b63f02962ea1 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:48.535840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72d4ec-0f9d-4cd3-b86c-b63f02962ea1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:48.537756 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72d4ec-0f9d-4cd3-b86c-b63f02962ea1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:48.559101 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1356/2735] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:44:48 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:44:48.563012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50e5c5-5515-4df4-a584-3243f20909c1 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:48.573482 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50e5c5-5515-4df4-a584-3243f20909c1 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:48.578103 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3723be5-b826-4495-8900-492a74790453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:48.578718 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3723be5-b826-4495-8900-492a74790453 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:48.578994 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3723be5-b826-4495-8900-492a74790453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:48.579264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3723be5-b826-4495-8900-492a74790453 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:48.579979 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3723be5-b826-4495-8900-492a74790453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:48.581546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3723be5-b826-4495-8900-492a74790453 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:48.591454 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1380/2736] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:44:48.595493 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72d4ec-0f9d-4cd3-b86c-b63f02962ea1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:48.605643 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72d4ec-0f9d-4cd3-b86c-b63f02962ea1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:48.610175 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ebbb8-c836-438d-9f0a-015ce4d628c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:48.610830 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ebbb8-c836-438d-9f0a-015ce4d628c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:48.611108 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ebbb8-c836-438d-9f0a-015ce4d628c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:48.611379 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ebbb8-c836-438d-9f0a-015ce4d628c3 admin admin] PATH_INFO: `/v3/projects/b8473f1322934b92bf52f72d90ebdb80/users/31e585172abe4750b80422187dca0395/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:48.626142 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43ebbb8-c836-438d-9f0a-015ce4d628c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8473f1322934b92bf52f72d90ebdb80, user_id=31e585172abe4750b80422187dca0395, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:48.628830 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43ebbb8-c836-438d-9f0a-015ce4d628c3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:48.642181 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1357/2737] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:44:48 2022] PUT /identity/v3/projects/b8473f1322934b92bf52f72d90ebdb80/users/31e585172abe4750b80422187dca0395/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:44:48.646087 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3723be5-b826-4495-8900-492a74790453 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:48.656219 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3723be5-b826-4495-8900-492a74790453 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:48.712095 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e96fa-f6ec-40de-a61a-7b7324a6a6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:48.712607 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e96fa-f6ec-40de-a61a-7b7324a6a6a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:48.712891 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e96fa-f6ec-40de-a61a-7b7324a6a6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:48.713163 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e96fa-f6ec-40de-a61a-7b7324a6a6a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:48.713799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e96fa-f6ec-40de-a61a-7b7324a6a6a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:48.715947 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e96fa-f6ec-40de-a61a-7b7324a6a6a9 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:48.725412 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1381/2738] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:44:48.729510 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ebbb8-c836-438d-9f0a-015ce4d628c3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:48.740660 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ebbb8-c836-438d-9f0a-015ce4d628c3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:48.745294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a16c5d-5ea1-4803-a5ea-6d897678c5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:48.745923 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a16c5d-5ea1-4803-a5ea-6d897678c5ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:48.746225 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a16c5d-5ea1-4803-a5ea-6d897678c5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:48.746507 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a16c5d-5ea1-4803-a5ea-6d897678c5ea admin admin] PATH_INFO: `/v3/projects/b8473f1322934b92bf52f72d90ebdb80/users/31e585172abe4750b80422187dca0395/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:48.748417 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a16c5d-5ea1-4803-a5ea-6d897678c5ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8473f1322934b92bf52f72d90ebdb80, user_id=31e585172abe4750b80422187dca0395, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:48.751072 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a16c5d-5ea1-4803-a5ea-6d897678c5ea admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:48.761125 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1358/2739] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:44:48 2022] PUT /identity/v3/projects/b8473f1322934b92bf52f72d90ebdb80/users/31e585172abe4750b80422187dca0395/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:48.771971 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32902a19-c771-43ac-afef-ae7c3ceaddb4 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:48.772262 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32902a19-c771-43ac-afef-ae7c3ceaddb4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:48.772541 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32902a19-c771-43ac-afef-ae7c3ceaddb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:48.800815 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-32902a19-c771-43ac-afef-ae7c3ceaddb4 None None] MFA Rules not processed for user `31e585172abe4750b80422187dca0395`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:48.802005 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-32902a19-c771-43ac-afef-ae7c3ceaddb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:48.869799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1382/2740] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:44:48 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:48.883390 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a16c5d-5ea1-4803-a5ea-6d897678c5ea admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:48.885773 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e9a16c5d-5ea1-4803-a5ea-6d897678c5ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:48.948779 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a16c5d-5ea1-4803-a5ea-6d897678c5ea admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:48.952867 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732eec04-a1f0-4972-bb18-7ae5b6e9e507 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:48.953664 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732eec04-a1f0-4972-bb18-7ae5b6e9e507 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:48.953985 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732eec04-a1f0-4972-bb18-7ae5b6e9e507 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:48.954293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732eec04-a1f0-4972-bb18-7ae5b6e9e507 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:48.963715 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732eec04-a1f0-4972-bb18-7ae5b6e9e507 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:48.965615 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732eec04-a1f0-4972-bb18-7ae5b6e9e507 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:48.975437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1359/2741] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:48 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:44:49.398154 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32902a19-c771-43ac-afef-ae7c3ceaddb4 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:49.449921 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:49.463660 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32902a19-c771-43ac-afef-ae7c3ceaddb4 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:49.468901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427bb912-ccbb-4d52-8c71-46f7398058a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:49.469542 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bb912-ccbb-4d52-8c71-46f7398058a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:49.469820 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bb912-ccbb-4d52-8c71-46f7398058a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:49.470092 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bb912-ccbb-4d52-8c71-46f7398058a8 admin admin] PATH_INFO: `/v3/users/8404a4ada2084aceb0019d51855c71c0` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:49.471246 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427bb912-ccbb-4d52-8c71-46f7398058a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8404a4ada2084aceb0019d51855c71c0)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:49.473152 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427bb912-ccbb-4d52-8c71-46f7398058a8 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:49.520714 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-427bb912-ccbb-4d52-8c71-46f7398058a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8404a4ada2084aceb0019d51855c71c0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:49.528138 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-427bb912-ccbb-4d52-8c71-46f7398058a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8404a4ada2084aceb0019d51855c71c0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:49.528829 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-427bb912-ccbb-4d52-8c71-46f7398058a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8404a4ada2084aceb0019d51855c71c0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:49.529432 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-427bb912-ccbb-4d52-8c71-46f7398058a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8404a4ada2084aceb0019d51855c71c0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:49.541938 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-427bb912-ccbb-4d52-8c71-46f7398058a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8404a4ada2084aceb0019d51855c71c0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:49.558944 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1383/2742] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:44:49 2022] DELETE /identity/v3/users/8404a4ada2084aceb0019d51855c71c0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:44:49.561223 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529213dd-7f6c-4b32-b67b-090a42d1a700 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:49.561761 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529213dd-7f6c-4b32-b67b-090a42d1a700 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:49.561987 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529213dd-7f6c-4b32-b67b-090a42d1a700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:49.581894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-529213dd-7f6c-4b32-b67b-090a42d1a700 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:49.583147 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-529213dd-7f6c-4b32-b67b-090a42d1a700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:49.621947 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:49.658506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1360/2743] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:44:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:44:49.669814 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427bb912-ccbb-4d52-8c71-46f7398058a8 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:49.671377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-427bb912-ccbb-4d52-8c71-46f7398058a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:49.699629 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529213dd-7f6c-4b32-b67b-090a42d1a700 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:49.701803 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-529213dd-7f6c-4b32-b67b-090a42d1a700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:49.721925 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529213dd-7f6c-4b32-b67b-090a42d1a700 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:49.727637 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ae03d-4906-41d2-b736-79b1f40a7de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:49.728283 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ae03d-4906-41d2-b736-79b1f40a7de2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:49.728572 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ae03d-4906-41d2-b736-79b1f40a7de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:49.728854 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ae03d-4906-41d2-b736-79b1f40a7de2 admin admin] PATH_INFO: `/v3/users/3fbb843ada72400e9d9f06521c2a61ab` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:49.730094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ae03d-4906-41d2-b736-79b1f40a7de2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fbb843ada72400e9d9f06521c2a61ab)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:49.732592 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ae03d-4906-41d2-b736-79b1f40a7de2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:49.739987 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427bb912-ccbb-4d52-8c71-46f7398058a8 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:49.743924 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbdaa13-9f22-49b1-a10d-4e58e6c3e9fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:49.744604 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbdaa13-9f22-49b1-a10d-4e58e6c3e9fc service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:49.744953 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbdaa13-9f22-49b1-a10d-4e58e6c3e9fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:49.745547 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbdaa13-9f22-49b1-a10d-4e58e6c3e9fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:49.755298 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbdaa13-9f22-49b1-a10d-4e58e6c3e9fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:49.756803 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbdaa13-9f22-49b1-a10d-4e58e6c3e9fc service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:49.768760 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1384/2744] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:44:49.787320 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-929ae03d-4906-41d2-b736-79b1f40a7de2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fbb843ada72400e9d9f06521c2a61ab'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:49.795220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-929ae03d-4906-41d2-b736-79b1f40a7de2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fbb843ada72400e9d9f06521c2a61ab'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:49.796073 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-929ae03d-4906-41d2-b736-79b1f40a7de2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fbb843ada72400e9d9f06521c2a61ab'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:49.805333 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-929ae03d-4906-41d2-b736-79b1f40a7de2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fbb843ada72400e9d9f06521c2a61ab'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:49.821054 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-929ae03d-4906-41d2-b736-79b1f40a7de2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fbb843ada72400e9d9f06521c2a61ab'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:49.829324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1361/2745] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:44:49 2022] DELETE /identity/v3/users/3fbb843ada72400e9d9f06521c2a61ab => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:44:50.051169 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbdaa13-9f22-49b1-a10d-4e58e6c3e9fc service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:50.070186 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbdaa13-9f22-49b1-a10d-4e58e6c3e9fc service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:50.086138 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ae03d-4906-41d2-b736-79b1f40a7de2 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:50.087914 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-929ae03d-4906-41d2-b736-79b1f40a7de2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:50.140044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7af18d-6add-41ef-96db-f479fe42d12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:50.140676 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7af18d-6add-41ef-96db-f479fe42d12c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:50.140952 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7af18d-6add-41ef-96db-f479fe42d12c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:50.141226 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7af18d-6add-41ef-96db-f479fe42d12c admin admin] PATH_INFO: `/v3/projects/ca311e408a714ede8d413375ca68eb67` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:50.142390 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7af18d-6add-41ef-96db-f479fe42d12c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca311e408a714ede8d413375ca68eb67)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:50.144320 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7af18d-6add-41ef-96db-f479fe42d12c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:50.159887 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:50.160075 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ae03d-4906-41d2-b736-79b1f40a7de2 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:50.165679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a304fac9-9cf4-45b9-9a9f-0a2ca243e022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:50.166398 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a304fac9-9cf4-45b9-9a9f-0a2ca243e022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:50.166684 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a304fac9-9cf4-45b9-9a9f-0a2ca243e022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:50.166970 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a304fac9-9cf4-45b9-9a9f-0a2ca243e022 admin admin] PATH_INFO: `/v3/projects/d17194312d83471f9c25716be39139ce` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:50.168832 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a304fac9-9cf4-45b9-9a9f-0a2ca243e022 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d17194312d83471f9c25716be39139ce)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:50.170881 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a304fac9-9cf4-45b9-9a9f-0a2ca243e022 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:50.193405 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6e7af18d-6add-41ef-96db-f479fe42d12c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca311e408a714ede8d413375ca68eb67', 'request_id': 'req-6e7af18d-6add-41ef-96db-f479fe42d12c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:50.200039 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6e7af18d-6add-41ef-96db-f479fe42d12c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca311e408a714ede8d413375ca68eb67', 'request_id': 'req-6e7af18d-6add-41ef-96db-f479fe42d12c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:50.206566 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6e7af18d-6add-41ef-96db-f479fe42d12c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca311e408a714ede8d413375ca68eb67', 'request_id': 'req-6e7af18d-6add-41ef-96db-f479fe42d12c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:50.219297 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6e7af18d-6add-41ef-96db-f479fe42d12c admin admin] The token cache is being invalidate because project ca311e408a714ede8d413375ca68eb67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:44:50.219580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6e7af18d-6add-41ef-96db-f479fe42d12c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:50.220049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:50.220289 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6e7af18d-6add-41ef-96db-f479fe42d12c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:50.221751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1385/2746] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:44:50 2022] DELETE /identity/v3/projects/ca311e408a714ede8d413375ca68eb67 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:50.225732 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a304fac9-9cf4-45b9-9a9f-0a2ca243e022 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd17194312d83471f9c25716be39139ce', 'request_id': 'req-a304fac9-9cf4-45b9-9a9f-0a2ca243e022'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:50.226603 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6bc10-d7a8-4098-ac30-678bf56fdbe7 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:50.226887 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6bc10-d7a8-4098-ac30-678bf56fdbe7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:50.227163 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6bc10-d7a8-4098-ac30-678bf56fdbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:50.233594 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a304fac9-9cf4-45b9-9a9f-0a2ca243e022 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd17194312d83471f9c25716be39139ce', 'request_id': 'req-a304fac9-9cf4-45b9-9a9f-0a2ca243e022'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:50.242925 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a304fac9-9cf4-45b9-9a9f-0a2ca243e022 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd17194312d83471f9c25716be39139ce', 'request_id': 'req-a304fac9-9cf4-45b9-9a9f-0a2ca243e022'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:50.248927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-6aa6bc10-d7a8-4098-ac30-678bf56fdbe7 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:50.250137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6aa6bc10-d7a8-4098-ac30-678bf56fdbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:50.254294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a304fac9-9cf4-45b9-9a9f-0a2ca243e022 admin admin] The token cache is being invalidate because project d17194312d83471f9c25716be39139ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:44:50.254851 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a304fac9-9cf4-45b9-9a9f-0a2ca243e022 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:50.255589 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a304fac9-9cf4-45b9-9a9f-0a2ca243e022 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:50.257256 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1362/2747] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:44:50 2022] DELETE /identity/v3/projects/d17194312d83471f9c25716be39139ce => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:50.261256 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a304fac9-9cf4-45b9-9a9f-0a2ca243e022 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:50.262601 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a304fac9-9cf4-45b9-9a9f-0a2ca243e022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:50.314739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1386/2748] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:44:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:50.327256 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a304fac9-9cf4-45b9-9a9f-0a2ca243e022 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:50.332041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e8908-3665-4262-9394-d5fd63be045a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:50.332702 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e8908-3665-4262-9394-d5fd63be045a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:50.332982 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e8908-3665-4262-9394-d5fd63be045a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:50.333254 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e8908-3665-4262-9394-d5fd63be045a admin admin] PATH_INFO: `/v3/users/31e585172abe4750b80422187dca0395` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:50.334483 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e8908-3665-4262-9394-d5fd63be045a admin admin] RBAC: Authorizing `identity:delete_user(user_id=31e585172abe4750b80422187dca0395)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:50.336477 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e8908-3665-4262-9394-d5fd63be045a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:50.377281 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da685fc-0aad-4055-ac36-c9a68ed8f91d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:50.377556 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da685fc-0aad-4055-ac36-c9a68ed8f91d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:50.377826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da685fc-0aad-4055-ac36-c9a68ed8f91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:50.389981 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7d3e8908-3665-4262-9394-d5fd63be045a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31e585172abe4750b80422187dca0395'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:50.396878 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-6da685fc-0aad-4055-ac36-c9a68ed8f91d None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:50.398086 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6da685fc-0aad-4055-ac36-c9a68ed8f91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:50.399197 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7d3e8908-3665-4262-9394-d5fd63be045a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31e585172abe4750b80422187dca0395'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:50.399901 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7d3e8908-3665-4262-9394-d5fd63be045a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31e585172abe4750b80422187dca0395'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:50.406532 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1387/2749] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:44:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:44:50.408442 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7d3e8908-3665-4262-9394-d5fd63be045a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31e585172abe4750b80422187dca0395'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:50.411059 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da685fc-0aad-4055-ac36-c9a68ed8f91d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:50.421353 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da685fc-0aad-4055-ac36-c9a68ed8f91d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:50.425461 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7d3e8908-3665-4262-9394-d5fd63be045a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31e585172abe4750b80422187dca0395'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:50.426111 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1360e-06a6-4f23-a29d-52adcfb4ab0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:50.426797 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1360e-06a6-4f23-a29d-52adcfb4ab0d admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:50.427073 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1360e-06a6-4f23-a29d-52adcfb4ab0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:50.427344 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1360e-06a6-4f23-a29d-52adcfb4ab0d admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:50.427989 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb1360e-06a6-4f23-a29d-52adcfb4ab0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:44:50.428294 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb1360e-06a6-4f23-a29d-52adcfb4ab0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:50.429867 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb1360e-06a6-4f23-a29d-52adcfb4ab0d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:50.432307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1363/2750] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:44:50 2022] DELETE /identity/v3/users/31e585172abe4750b80422187dca0395 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:50.443421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1388/2751] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:44:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:44:50.447832 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e8908-3665-4262-9394-d5fd63be045a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:50.449334 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7d3e8908-3665-4262-9394-d5fd63be045a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:50.513009 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:50.518634 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e8908-3665-4262-9394-d5fd63be045a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:50.524979 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8119d-036f-4880-a087-b692bd102d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:50.525601 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8119d-036f-4880-a087-b692bd102d20 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:50.526102 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8119d-036f-4880-a087-b692bd102d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:50.526491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8119d-036f-4880-a087-b692bd102d20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:50.528304 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8119d-036f-4880-a087-b692bd102d20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:50.530274 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8119d-036f-4880-a087-b692bd102d20 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:50.560058 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1364/2752] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:44:50 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:44:50.564492 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1360e-06a6-4f23-a29d-52adcfb4ab0d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:50.575640 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1360e-06a6-4f23-a29d-52adcfb4ab0d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:50.639119 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb639f90-6a3e-42af-a470-7c25a28ed2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:50.639747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb639f90-6a3e-42af-a470-7c25a28ed2de admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:50.640070 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb639f90-6a3e-42af-a470-7c25a28ed2de admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:50.640351 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb639f90-6a3e-42af-a470-7c25a28ed2de admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:50.641004 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb639f90-6a3e-42af-a470-7c25a28ed2de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:50.642972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb639f90-6a3e-42af-a470-7c25a28ed2de admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:50.667050 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1389/2753] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:44:50 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:44:50.671213 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8119d-036f-4880-a087-b692bd102d20 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:50.681721 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8119d-036f-4880-a087-b692bd102d20 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:50.687015 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230c42c-7fac-4e12-905f-69ac632e1594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:50.687862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230c42c-7fac-4e12-905f-69ac632e1594 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:50.688151 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230c42c-7fac-4e12-905f-69ac632e1594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:50.688430 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230c42c-7fac-4e12-905f-69ac632e1594 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:50.689089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a230c42c-7fac-4e12-905f-69ac632e1594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:50.690942 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a230c42c-7fac-4e12-905f-69ac632e1594 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:50.703471 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1365/2754] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:44:50.707799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb639f90-6a3e-42af-a470-7c25a28ed2de admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:50.719740 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb639f90-6a3e-42af-a470-7c25a28ed2de admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:50.725102 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230c42c-7fac-4e12-905f-69ac632e1594 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:50.726627 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2120c-432c-4440-81ae-b5f2d28e55fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:50.727268 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2120c-432c-4440-81ae-b5f2d28e55fe admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:50.727552 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2120c-432c-4440-81ae-b5f2d28e55fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:50.727826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2120c-432c-4440-81ae-b5f2d28e55fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:50.728540 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f2120c-432c-4440-81ae-b5f2d28e55fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:50.730101 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a230c42c-7fac-4e12-905f-69ac632e1594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:50.730496 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f2120c-432c-4440-81ae-b5f2d28e55fe admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:50.742656 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1390/2755] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:44:50.746417 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2120c-432c-4440-81ae-b5f2d28e55fe admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:50.751660 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:50.753735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230c42c-7fac-4e12-905f-69ac632e1594 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:50.757604 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2120c-432c-4440-81ae-b5f2d28e55fe admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:50.758739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929021a0-2096-47f7-befc-1ccd5d2872ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:50.759323 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929021a0-2096-47f7-befc-1ccd5d2872ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:50.759571 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929021a0-2096-47f7-befc-1ccd5d2872ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:50.759847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929021a0-2096-47f7-befc-1ccd5d2872ef admin admin] PATH_INFO: `/v3/projects/b8473f1322934b92bf52f72d90ebdb80` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:50.761675 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929021a0-2096-47f7-befc-1ccd5d2872ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8473f1322934b92bf52f72d90ebdb80)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:50.764048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929021a0-2096-47f7-befc-1ccd5d2872ef admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:50.764267 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5f3724-dfb2-461b-bc22-d25635007dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:50.764916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f3724-dfb2-461b-bc22-d25635007dda admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:50.765318 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f3724-dfb2-461b-bc22-d25635007dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:50.765594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f3724-dfb2-461b-bc22-d25635007dda admin admin] PATH_INFO: `/v3/projects/66de2555209e4af099f1757d69bfa70d/users/0b2181102c4d42af80664d12923cabbe/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:50.784486 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5f3724-dfb2-461b-bc22-d25635007dda admin admin] RBAC: Authorizing `identity:create_grant(project_id=66de2555209e4af099f1757d69bfa70d, user_id=0b2181102c4d42af80664d12923cabbe, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:50.787138 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5f3724-dfb2-461b-bc22-d25635007dda admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:50.800360 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1391/2756] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:44:50 2022] PUT /identity/v3/projects/66de2555209e4af099f1757d69bfa70d/users/0b2181102c4d42af80664d12923cabbe/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:44:50.810248 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfea6d9-0067-4347-bc0d-3024cec7c35a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:50.810543 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfea6d9-0067-4347-bc0d-3024cec7c35a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:50.810820 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfea6d9-0067-4347-bc0d-3024cec7c35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:50.820514 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-929021a0-2096-47f7-befc-1ccd5d2872ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8473f1322934b92bf52f72d90ebdb80', 'request_id': 'req-929021a0-2096-47f7-befc-1ccd5d2872ef'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:50.827881 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-929021a0-2096-47f7-befc-1ccd5d2872ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8473f1322934b92bf52f72d90ebdb80', 'request_id': 'req-929021a0-2096-47f7-befc-1ccd5d2872ef'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:50.836789 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-929021a0-2096-47f7-befc-1ccd5d2872ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8473f1322934b92bf52f72d90ebdb80', 'request_id': 'req-929021a0-2096-47f7-befc-1ccd5d2872ef'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:50.841088 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-8bfea6d9-0067-4347-bc0d-3024cec7c35a None None] MFA Rules not processed for user `0b2181102c4d42af80664d12923cabbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:50.842497 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8bfea6d9-0067-4347-bc0d-3024cec7c35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:50.848273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-929021a0-2096-47f7-befc-1ccd5d2872ef admin admin] The token cache is being invalidate because project b8473f1322934b92bf52f72d90ebdb80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:44:50.848565 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-929021a0-2096-47f7-befc-1ccd5d2872ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:50.849210 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-929021a0-2096-47f7-befc-1ccd5d2872ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:50.850952 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1366/2757] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:44:50 2022] DELETE /identity/v3/projects/b8473f1322934b92bf52f72d90ebdb80 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:44:50.855638 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb8088-291c-45fe-9b30-a92dc3141f09 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:50.856172 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb8088-291c-45fe-9b30-a92dc3141f09 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:50.856548 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb8088-291c-45fe-9b30-a92dc3141f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:50.875921 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c3bb8088-291c-45fe-9b30-a92dc3141f09 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:50.877257 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c3bb8088-291c-45fe-9b30-a92dc3141f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:50.908264 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1392/2758] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:44:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:44:50.919799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfea6d9-0067-4347-bc0d-3024cec7c35a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:50.921153 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8bfea6d9-0067-4347-bc0d-3024cec7c35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:50.937388 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1367/2759] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:44:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:44:50.944983 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ca842-d79f-45cc-96fb-ab9ce1551dc9 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:50.945297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ca842-d79f-45cc-96fb-ab9ce1551dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:50.945574 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ca842-d79f-45cc-96fb-ab9ce1551dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:50.965546 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-818ca842-d79f-45cc-96fb-ab9ce1551dc9 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:50.966762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-818ca842-d79f-45cc-96fb-ab9ce1551dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:50.974466 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1368/2760] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:44:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:44:50.977573 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfea6d9-0067-4347-bc0d-3024cec7c35a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:50.978536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818ca842-d79f-45cc-96fb-ab9ce1551dc9 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:50.981554 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33294b43-05ce-4a23-9e68-568640dd7bdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:50.982173 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33294b43-05ce-4a23-9e68-568640dd7bdd service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:50.982482 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33294b43-05ce-4a23-9e68-568640dd7bdd service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:50.982877 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33294b43-05ce-4a23-9e68-568640dd7bdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:50.987477 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818ca842-d79f-45cc-96fb-ab9ce1551dc9 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:50.991406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33294b43-05ce-4a23-9e68-568640dd7bdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:50.992011 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e618a07-5874-460a-9ea0-973185b5485d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:50.992588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e618a07-5874-460a-9ea0-973185b5485d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:50.992825 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33294b43-05ce-4a23-9e68-568640dd7bdd service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:50.992929 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e618a07-5874-460a-9ea0-973185b5485d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:50.993127 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e618a07-5874-460a-9ea0-973185b5485d admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:50.993764 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e618a07-5874-460a-9ea0-973185b5485d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:44:50.994065 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e618a07-5874-460a-9ea0-973185b5485d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:50.996198 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e618a07-5874-460a-9ea0-973185b5485d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:51.002267 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1393/2761] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:44:51.007776 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1369/2762] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:44:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:44:51.012688 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33294b43-05ce-4a23-9e68-568640dd7bdd service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:51.023161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33294b43-05ce-4a23-9e68-568640dd7bdd service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:51.027821 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df987f91-7327-440e-8758-196157004a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:51.028410 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df987f91-7327-440e-8758-196157004a23 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:51.028693 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df987f91-7327-440e-8758-196157004a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:51.028969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df987f91-7327-440e-8758-196157004a23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:51.029611 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df987f91-7327-440e-8758-196157004a23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:51.031520 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df987f91-7327-440e-8758-196157004a23 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:51.061310 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1394/2763] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:44:51 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:44:51.065776 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e618a07-5874-460a-9ea0-973185b5485d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:51.076128 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e618a07-5874-460a-9ea0-973185b5485d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:51.107360 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:51.139639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2897e-dbd8-4497-8ebb-b04f54405554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:51.140317 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2897e-dbd8-4497-8ebb-b04f54405554 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:51.140604 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2897e-dbd8-4497-8ebb-b04f54405554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:51.140885 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2897e-dbd8-4497-8ebb-b04f54405554 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:51.142153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe2897e-dbd8-4497-8ebb-b04f54405554 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:51.144099 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe2897e-dbd8-4497-8ebb-b04f54405554 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:51.168016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1370/2764] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:44:51 2022] POST /identity/v3/users => generated 286 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:44:51.172380 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df987f91-7327-440e-8758-196157004a23 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:51.184911 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df987f91-7327-440e-8758-196157004a23 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:51.189943 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40226b90-c0e1-46e3-a23c-47b251e8435b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:51.190750 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40226b90-c0e1-46e3-a23c-47b251e8435b admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:51.191184 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40226b90-c0e1-46e3-a23c-47b251e8435b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:51.191500 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40226b90-c0e1-46e3-a23c-47b251e8435b admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:51.192403 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40226b90-c0e1-46e3-a23c-47b251e8435b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:51.194044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40226b90-c0e1-46e3-a23c-47b251e8435b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:51.204760 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1395/2765] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:44:51.208764 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2897e-dbd8-4497-8ebb-b04f54405554 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:51.220273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2897e-dbd8-4497-8ebb-b04f54405554 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:51.226141 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2569714a-c743-4189-98fb-ec3a6722b4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:51.226772 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569714a-c743-4189-98fb-ec3a6722b4e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:51.227060 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569714a-c743-4189-98fb-ec3a6722b4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:51.227434 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569714a-c743-4189-98fb-ec3a6722b4e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:51.227970 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2569714a-c743-4189-98fb-ec3a6722b4e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:51.229568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2569714a-c743-4189-98fb-ec3a6722b4e4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:51.243446 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1371/2766] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:44:51.248052 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40226b90-c0e1-46e3-a23c-47b251e8435b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:51.259247 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40226b90-c0e1-46e3-a23c-47b251e8435b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:51.268655 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef90af1-1f23-4440-b290-820e44d757c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:51.269338 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef90af1-1f23-4440-b290-820e44d757c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:51.269614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef90af1-1f23-4440-b290-820e44d757c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:51.269888 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef90af1-1f23-4440-b290-820e44d757c6 admin admin] PATH_INFO: `/v3/projects/0c5f5927d8cf48699c0691b46f9c461f/users/f833dec98cc74e89a5f317a572357e0c/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:51.287378 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef90af1-1f23-4440-b290-820e44d757c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c5f5927d8cf48699c0691b46f9c461f, user_id=f833dec98cc74e89a5f317a572357e0c, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:51.290263 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef90af1-1f23-4440-b290-820e44d757c6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:51.310916 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1396/2767] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:44:51 2022] PUT /identity/v3/projects/0c5f5927d8cf48699c0691b46f9c461f/users/f833dec98cc74e89a5f317a572357e0c/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:44:51.312654 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79347d-7f49-4bd1-9d99-822c2dacfb4f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:51.313365 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79347d-7f49-4bd1-9d99-822c2dacfb4f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:51.313825 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79347d-7f49-4bd1-9d99-822c2dacfb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:51.335509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-cc79347d-7f49-4bd1-9d99-822c2dacfb4f None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:51.336875 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cc79347d-7f49-4bd1-9d99-822c2dacfb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:51.399197 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1372/2768] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:44:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:51.408845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef90af1-1f23-4440-b290-820e44d757c6 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:51.410577 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bef90af1-1f23-4440-b290-820e44d757c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:51.425957 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:51.470513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef90af1-1f23-4440-b290-820e44d757c6 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:51.474134 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f555d33-82c3-4398-babe-94576005983c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:51.474760 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f555d33-82c3-4398-babe-94576005983c service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:51.475035 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f555d33-82c3-4398-babe-94576005983c service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:51.475304 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f555d33-82c3-4398-babe-94576005983c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:51.484777 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f555d33-82c3-4398-babe-94576005983c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:51.486176 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f555d33-82c3-4398-babe-94576005983c service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:51.495356 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1397/2769] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:44:52.097044 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3376826-babe-4ace-8315-716c7c10e3cb None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:52.097615 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3376826-babe-4ace-8315-716c7c10e3cb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:52.097728 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3376826-babe-4ace-8315-716c7c10e3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:52.126611 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d3376826-babe-4ace-8315-716c7c10e3cb None None] MFA Rules not processed for user `f833dec98cc74e89a5f317a572357e0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:52.127939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d3376826-babe-4ace-8315-716c7c10e3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:52.197558 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1373/2770] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:44:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:52.202580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f555d33-82c3-4398-babe-94576005983c service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:52.212064 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f555d33-82c3-4398-babe-94576005983c service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:52.217513 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522094e-1ed7-4f1c-a975-c64230dc3278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:52.218115 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522094e-1ed7-4f1c-a975-c64230dc3278 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:52.218439 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522094e-1ed7-4f1c-a975-c64230dc3278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:52.218713 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522094e-1ed7-4f1c-a975-c64230dc3278 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:52.219358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522094e-1ed7-4f1c-a975-c64230dc3278 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:52.221423 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522094e-1ed7-4f1c-a975-c64230dc3278 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:52.251899 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1398/2771] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:44:52 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:44:52.255849 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3376826-babe-4ace-8315-716c7c10e3cb None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:52.267487 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3376826-babe-4ace-8315-716c7c10e3cb None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:52.311555 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:52.330676 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c3746-347d-4670-bf97-2663038f86fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:52.331310 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c3746-347d-4670-bf97-2663038f86fb admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:52.331592 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c3746-347d-4670-bf97-2663038f86fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:52.331967 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c3746-347d-4670-bf97-2663038f86fb admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:52.332615 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c3746-347d-4670-bf97-2663038f86fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:52.334629 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c3746-347d-4670-bf97-2663038f86fb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:52.356748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1374/2772] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:44:52 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:44:52.360834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522094e-1ed7-4f1c-a975-c64230dc3278 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:52.371878 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522094e-1ed7-4f1c-a975-c64230dc3278 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:52.376678 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeedfb96-c7ee-4432-9efb-57c0132d24cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:52.377346 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeedfb96-c7ee-4432-9efb-57c0132d24cb admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:52.377696 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeedfb96-c7ee-4432-9efb-57c0132d24cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:52.378037 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeedfb96-c7ee-4432-9efb-57c0132d24cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:52.378787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeedfb96-c7ee-4432-9efb-57c0132d24cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:52.380787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeedfb96-c7ee-4432-9efb-57c0132d24cb admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:52.392387 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1399/2773] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:44:52.396619 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c3746-347d-4670-bf97-2663038f86fb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:52.408430 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c3746-347d-4670-bf97-2663038f86fb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:52.412703 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4b1db-2078-406c-a111-f2aa1141e710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:52.413378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4b1db-2078-406c-a111-f2aa1141e710 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:52.413658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4b1db-2078-406c-a111-f2aa1141e710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:52.413929 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4b1db-2078-406c-a111-f2aa1141e710 admin admin] PATH_INFO: `/v3/projects/2314b005429e473397437b879deaef43/users/ca20f893b7a84121a4ed22499c34c85b/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:52.429075 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef4b1db-2078-406c-a111-f2aa1141e710 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2314b005429e473397437b879deaef43, user_id=ca20f893b7a84121a4ed22499c34c85b, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:52.431724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef4b1db-2078-406c-a111-f2aa1141e710 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:52.446367 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1375/2774] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:44:52 2022] PUT /identity/v3/projects/2314b005429e473397437b879deaef43/users/ca20f893b7a84121a4ed22499c34c85b/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:52.450502 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeedfb96-c7ee-4432-9efb-57c0132d24cb admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:52.461904 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeedfb96-c7ee-4432-9efb-57c0132d24cb admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:52.526103 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f361c-27ad-455b-ba55-266068c21233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:52.526814 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f361c-27ad-455b-ba55-266068c21233 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:52.527257 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f361c-27ad-455b-ba55-266068c21233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:52.527598 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f361c-27ad-455b-ba55-266068c21233 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:52.528352 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280f361c-27ad-455b-ba55-266068c21233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:52.529939 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280f361c-27ad-455b-ba55-266068c21233 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:52.540662 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1400/2775] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:44:52.545524 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4b1db-2078-406c-a111-f2aa1141e710 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:52.555757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4b1db-2078-406c-a111-f2aa1141e710 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:52.560982 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7e353-253b-4a88-8325-47277d764a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:52.561635 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7e353-253b-4a88-8325-47277d764a70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:52.561918 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7e353-253b-4a88-8325-47277d764a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:52.562217 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7e353-253b-4a88-8325-47277d764a70 admin admin] PATH_INFO: `/v3/projects/2314b005429e473397437b879deaef43/users/ca20f893b7a84121a4ed22499c34c85b/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:52.564199 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7e353-253b-4a88-8325-47277d764a70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2314b005429e473397437b879deaef43, user_id=ca20f893b7a84121a4ed22499c34c85b, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:52.567921 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7e353-253b-4a88-8325-47277d764a70 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:52.579447 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1376/2776] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:44:52 2022] PUT /identity/v3/projects/2314b005429e473397437b879deaef43/users/ca20f893b7a84121a4ed22499c34c85b/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:53.214121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22b400-ae4c-4566-8b91-12f7de5e7b33 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:53.215159 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22b400-ae4c-4566-8b91-12f7de5e7b33 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:53.215263 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22b400-ae4c-4566-8b91-12f7de5e7b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:53.243606 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-fe22b400-ae4c-4566-8b91-12f7de5e7b33 None None] MFA Rules not processed for user `ca20f893b7a84121a4ed22499c34c85b`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:53.244861 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-fe22b400-ae4c-4566-8b91-12f7de5e7b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:53.317245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1401/2777] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:44:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:53.328239 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7e353-253b-4a88-8325-47277d764a70 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:53.384702 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7e353-253b-4a88-8325-47277d764a70 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:53.388446 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6fd10c-ab53-4c32-ab81-fb116b8a1754 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:53.389069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6fd10c-ab53-4c32-ab81-fb116b8a1754 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:53.389353 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6fd10c-ab53-4c32-ab81-fb116b8a1754 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:53.389627 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6fd10c-ab53-4c32-ab81-fb116b8a1754 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:53.451622 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6fd10c-ab53-4c32-ab81-fb116b8a1754 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:53.453110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6fd10c-ab53-4c32-ab81-fb116b8a1754 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:53.462830 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1377/2778] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:44:53.585315 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22b400-ae4c-4566-8b91-12f7de5e7b33 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:53.586967 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-fe22b400-ae4c-4566-8b91-12f7de5e7b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:53.594628 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:53.668018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22b400-ae4c-4566-8b91-12f7de5e7b33 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:53.672776 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc272c2c-afc4-4a08-a0ab-c86dcba8216c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:53.673484 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc272c2c-afc4-4a08-a0ab-c86dcba8216c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:53.673762 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc272c2c-afc4-4a08-a0ab-c86dcba8216c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:53.674032 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc272c2c-afc4-4a08-a0ab-c86dcba8216c admin admin] PATH_INFO: `/v3/users/0b2181102c4d42af80664d12923cabbe` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:53.675182 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc272c2c-afc4-4a08-a0ab-c86dcba8216c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b2181102c4d42af80664d12923cabbe)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:53.677087 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc272c2c-afc4-4a08-a0ab-c86dcba8216c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:53.743245 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bc272c2c-afc4-4a08-a0ab-c86dcba8216c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b2181102c4d42af80664d12923cabbe'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:53.752367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bc272c2c-afc4-4a08-a0ab-c86dcba8216c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b2181102c4d42af80664d12923cabbe'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:53.753158 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bc272c2c-afc4-4a08-a0ab-c86dcba8216c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b2181102c4d42af80664d12923cabbe'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:53.753859 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bc272c2c-afc4-4a08-a0ab-c86dcba8216c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b2181102c4d42af80664d12923cabbe'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:53.770619 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bc272c2c-afc4-4a08-a0ab-c86dcba8216c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b2181102c4d42af80664d12923cabbe'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:53.791766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1402/2779] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:44:53 2022] DELETE /identity/v3/users/0b2181102c4d42af80664d12923cabbe => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:53.796055 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9c0e2-baee-48c7-ab90-4bd2768b71a5 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:53.796339 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9c0e2-baee-48c7-ab90-4bd2768b71a5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:53.796610 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9c0e2-baee-48c7-ab90-4bd2768b71a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:53.816769 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-cac9c0e2-baee-48c7-ab90-4bd2768b71a5 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:53.818056 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cac9c0e2-baee-48c7-ab90-4bd2768b71a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:53.863700 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:53.878878 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1378/2780] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:44:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:53.889083 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc272c2c-afc4-4a08-a0ab-c86dcba8216c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:53.890548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bc272c2c-afc4-4a08-a0ab-c86dcba8216c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:53.947011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc272c2c-afc4-4a08-a0ab-c86dcba8216c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:53.951051 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cced33d-081b-4e80-ba03-10b355cd4296 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:53.951641 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cced33d-081b-4e80-ba03-10b355cd4296 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:53.951913 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cced33d-081b-4e80-ba03-10b355cd4296 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:53.952183 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cced33d-081b-4e80-ba03-10b355cd4296 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:53.960867 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cced33d-081b-4e80-ba03-10b355cd4296 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:53.962306 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cced33d-081b-4e80-ba03-10b355cd4296 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:53.972318 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1403/2781] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:44:54.247751 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9c0e2-baee-48c7-ab90-4bd2768b71a5 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:54.249231 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cac9c0e2-baee-48c7-ab90-4bd2768b71a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:54.263580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9c0e2-baee-48c7-ab90-4bd2768b71a5 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:54.268342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa67e9b-d65c-417e-831c-9aba0c2609e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:54.268982 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa67e9b-d65c-417e-831c-9aba0c2609e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:54.269781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa67e9b-d65c-417e-831c-9aba0c2609e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:54.270065 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa67e9b-d65c-417e-831c-9aba0c2609e6 admin admin] PATH_INFO: `/v3/projects/66de2555209e4af099f1757d69bfa70d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:54.271442 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa67e9b-d65c-417e-831c-9aba0c2609e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66de2555209e4af099f1757d69bfa70d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:54.273353 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa67e9b-d65c-417e-831c-9aba0c2609e6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:54.323850 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-aaa67e9b-d65c-417e-831c-9aba0c2609e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66de2555209e4af099f1757d69bfa70d', 'request_id': 'req-aaa67e9b-d65c-417e-831c-9aba0c2609e6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:54.332414 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-aaa67e9b-d65c-417e-831c-9aba0c2609e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66de2555209e4af099f1757d69bfa70d', 'request_id': 'req-aaa67e9b-d65c-417e-831c-9aba0c2609e6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:54.341657 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-aaa67e9b-d65c-417e-831c-9aba0c2609e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66de2555209e4af099f1757d69bfa70d', 'request_id': 'req-aaa67e9b-d65c-417e-831c-9aba0c2609e6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:54.353392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-aaa67e9b-d65c-417e-831c-9aba0c2609e6 admin admin] The token cache is being invalidate because project 66de2555209e4af099f1757d69bfa70d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:44:54.353678 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-aaa67e9b-d65c-417e-831c-9aba0c2609e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:54.354325 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-aaa67e9b-d65c-417e-831c-9aba0c2609e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:54.356303 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1379/2782] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:44:54 2022] DELETE /identity/v3/projects/66de2555209e4af099f1757d69bfa70d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:54.361324 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19e835-4a59-4688-93bc-1e98a23fac6f None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:54.361843 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19e835-4a59-4688-93bc-1e98a23fac6f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:54.362417 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19e835-4a59-4688-93bc-1e98a23fac6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:54.383784 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2f19e835-4a59-4688-93bc-1e98a23fac6f None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:54.384975 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2f19e835-4a59-4688-93bc-1e98a23fac6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:54.404760 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:54.448210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1404/2783] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:44:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:44:54.455740 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a181d4-53c6-44eb-b8d6-2131516359ed None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:54.456030 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a181d4-53c6-44eb-b8d6-2131516359ed None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:54.456307 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a181d4-53c6-44eb-b8d6-2131516359ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:54.478749 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f0a181d4-53c6-44eb-b8d6-2131516359ed None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:54.479961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f0a181d4-53c6-44eb-b8d6-2131516359ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:54.488696 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1380/2784] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:44:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:54.492920 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f19e835-4a59-4688-93bc-1e98a23fac6f None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:54.503288 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f19e835-4a59-4688-93bc-1e98a23fac6f None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:54.507760 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9c3ed-09b1-48c1-8d66-eab8b0f775f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:54.508347 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9c3ed-09b1-48c1-8d66-eab8b0f775f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:54.508623 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9c3ed-09b1-48c1-8d66-eab8b0f775f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:54.508894 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9c3ed-09b1-48c1-8d66-eab8b0f775f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:54.509532 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c9c3ed-09b1-48c1-8d66-eab8b0f775f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:44:54.509833 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c9c3ed-09b1-48c1-8d66-eab8b0f775f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:54.511440 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c9c3ed-09b1-48c1-8d66-eab8b0f775f3 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:54.522623 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1405/2785] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:44:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:44:54.528336 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a181d4-53c6-44eb-b8d6-2131516359ed None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:54.537574 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a181d4-53c6-44eb-b8d6-2131516359ed None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:54.542451 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d924e-92f0-4b2e-8274-5d294a538aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:54.543597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d924e-92f0-4b2e-8274-5d294a538aa9 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:54.543819 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d924e-92f0-4b2e-8274-5d294a538aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:54.544094 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d924e-92f0-4b2e-8274-5d294a538aa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:54.544789 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d924e-92f0-4b2e-8274-5d294a538aa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:54.546780 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d924e-92f0-4b2e-8274-5d294a538aa9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:54.576089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1381/2786] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:44:54 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:44:54.580167 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9c3ed-09b1-48c1-8d66-eab8b0f775f3 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:54.590086 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9c3ed-09b1-48c1-8d66-eab8b0f775f3 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:54.651023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f7eb0-ef45-40a0-be3b-e69e81b97ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:54.651619 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f7eb0-ef45-40a0-be3b-e69e81b97ef6 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:54.651899 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f7eb0-ef45-40a0-be3b-e69e81b97ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:54.652166 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f7eb0-ef45-40a0-be3b-e69e81b97ef6 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:54.652783 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f7eb0-ef45-40a0-be3b-e69e81b97ef6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:54.654719 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f7eb0-ef45-40a0-be3b-e69e81b97ef6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:54.676125 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1406/2787] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:44:54 2022] POST /identity/v3/users => generated 282 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:44:54.680381 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d924e-92f0-4b2e-8274-5d294a538aa9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:54.691495 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d924e-92f0-4b2e-8274-5d294a538aa9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:54.696131 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1bc25a-b2c7-4441-9413-6cc013d22238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:54.696718 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1bc25a-b2c7-4441-9413-6cc013d22238 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:54.696995 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1bc25a-b2c7-4441-9413-6cc013d22238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:54.697265 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1bc25a-b2c7-4441-9413-6cc013d22238 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:54.697902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1bc25a-b2c7-4441-9413-6cc013d22238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:54.699537 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1bc25a-b2c7-4441-9413-6cc013d22238 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:54.709306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1382/2788] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:44:54.713184 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f7eb0-ef45-40a0-be3b-e69e81b97ef6 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:54.723936 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f7eb0-ef45-40a0-be3b-e69e81b97ef6 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:54.728548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645deb07-4714-475c-b80e-c32df5eba578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:54.729140 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645deb07-4714-475c-b80e-c32df5eba578 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:54.729424 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645deb07-4714-475c-b80e-c32df5eba578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:54.729703 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645deb07-4714-475c-b80e-c32df5eba578 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:54.731069 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645deb07-4714-475c-b80e-c32df5eba578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:54.732665 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645deb07-4714-475c-b80e-c32df5eba578 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:54.744124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1407/2789] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:44:54.748151 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1bc25a-b2c7-4441-9413-6cc013d22238 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:54.757881 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1bc25a-b2c7-4441-9413-6cc013d22238 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:54.762541 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab64270-62fb-4e63-a32b-cb5ee0376689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:54.763214 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab64270-62fb-4e63-a32b-cb5ee0376689 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:54.763491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab64270-62fb-4e63-a32b-cb5ee0376689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:54.763761 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab64270-62fb-4e63-a32b-cb5ee0376689 admin admin] PATH_INFO: `/v3/projects/03feb48be81c4865b12be5e67988a8d8/users/3fc79694d7734de08ae43e685a26138c/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:54.778433 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab64270-62fb-4e63-a32b-cb5ee0376689 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03feb48be81c4865b12be5e67988a8d8, user_id=3fc79694d7734de08ae43e685a26138c, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:54.781061 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab64270-62fb-4e63-a32b-cb5ee0376689 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:54.794216 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1383/2790] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:44:54 2022] PUT /identity/v3/projects/03feb48be81c4865b12be5e67988a8d8/users/3fc79694d7734de08ae43e685a26138c/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:54.803937 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca2c9e-1fa9-479e-adee-0c06f0443d53 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:54.804229 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca2c9e-1fa9-479e-adee-0c06f0443d53 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:54.804562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca2c9e-1fa9-479e-adee-0c06f0443d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:54.833088 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-51ca2c9e-1fa9-479e-adee-0c06f0443d53 None None] MFA Rules not processed for user `3fc79694d7734de08ae43e685a26138c`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:54.834289 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-51ca2c9e-1fa9-479e-adee-0c06f0443d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:54.856498 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab64270-62fb-4e63-a32b-cb5ee0376689 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:54.858288 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2ab64270-62fb-4e63-a32b-cb5ee0376689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:54.899159 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1408/2791] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:44:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:54.911684 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca2c9e-1fa9-479e-adee-0c06f0443d53 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:54.913238 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-51ca2c9e-1fa9-479e-adee-0c06f0443d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:54.915561 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab64270-62fb-4e63-a32b-cb5ee0376689 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:54.919781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8dbaa4-08cb-4b63-9a55-fc0fd56c83a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:54.920384 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8dbaa4-08cb-4b63-9a55-fc0fd56c83a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:54.920661 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8dbaa4-08cb-4b63-9a55-fc0fd56c83a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:54.920936 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8dbaa4-08cb-4b63-9a55-fc0fd56c83a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:54.922669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3b8dbaa4-08cb-4b63-9a55-fc0fd56c83a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:54.926137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca2c9e-1fa9-479e-adee-0c06f0443d53 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:54.929844 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd7280-4298-4f9f-98a1-6c2d463b10e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:54.930635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd7280-4298-4f9f-98a1-6c2d463b10e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:54.930919 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd7280-4298-4f9f-98a1-6c2d463b10e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:54.931194 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd7280-4298-4f9f-98a1-6c2d463b10e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:54.940514 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd7280-4298-4f9f-98a1-6c2d463b10e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:54.941926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd7280-4298-4f9f-98a1-6c2d463b10e4 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:54.944911 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:54.952133 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1409/2792] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:44:54.988327 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8dbaa4-08cb-4b63-9a55-fc0fd56c83a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:54.989813 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8dbaa4-08cb-4b63-9a55-fc0fd56c83a4 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:55.000270 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1384/2793] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:54 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:45:04.946701 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=93,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:04.995290 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=19 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:07.772319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd7280-4298-4f9f-98a1-6c2d463b10e4 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:07.773744 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a9fd7280-4298-4f9f-98a1-6c2d463b10e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:07.845752 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd7280-4298-4f9f-98a1-6c2d463b10e4 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:07.850844 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed994a9-651e-4707-bc29-5af66ee4b7bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12302cdb2854b0684179647a6dadf6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:07.851542 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed994a9-651e-4707-bc29-5af66ee4b7bc service placement] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:07.851858 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed994a9-651e-4707-bc29-5af66ee4b7bc service placement] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:07.852135 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed994a9-651e-4707-bc29-5af66ee4b7bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:07.853900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-eed994a9-651e-4707-bc29-5af66ee4b7bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:07.865161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed994a9-651e-4707-bc29-5af66ee4b7bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:07.866832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed994a9-651e-4707-bc29-5af66ee4b7bc service placement] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:07.876915 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1410/2794] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:12.287428 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8dbaa4-08cb-4b63-9a55-fc0fd56c83a4 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:12.289017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3b8dbaa4-08cb-4b63-9a55-fc0fd56c83a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:12.365992 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8dbaa4-08cb-4b63-9a55-fc0fd56c83a4 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:12.370794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9aa0a-f461-41fc-aa6a-a2052092e49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:12.371430 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9aa0a-f461-41fc-aa6a-a2052092e49d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:12.371716 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9aa0a-f461-41fc-aa6a-a2052092e49d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:12.372577 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9aa0a-f461-41fc-aa6a-a2052092e49d admin admin] PATH_INFO: `/v3/users/f833dec98cc74e89a5f317a572357e0c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:12.373730 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f9aa0a-f461-41fc-aa6a-a2052092e49d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f833dec98cc74e89a5f317a572357e0c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:12.375679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f9aa0a-f461-41fc-aa6a-a2052092e49d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:12.423728 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f1f9aa0a-f461-41fc-aa6a-a2052092e49d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f833dec98cc74e89a5f317a572357e0c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:12.430916 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f1f9aa0a-f461-41fc-aa6a-a2052092e49d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f833dec98cc74e89a5f317a572357e0c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:12.431594 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f1f9aa0a-f461-41fc-aa6a-a2052092e49d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f833dec98cc74e89a5f317a572357e0c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:12.440932 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f1f9aa0a-f461-41fc-aa6a-a2052092e49d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f833dec98cc74e89a5f317a572357e0c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:12.453303 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f1f9aa0a-f461-41fc-aa6a-a2052092e49d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f833dec98cc74e89a5f317a572357e0c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:12.459508 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1385/2795] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:45:12 2022] DELETE /identity/v3/users/f833dec98cc74e89a5f317a572357e0c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:12.630159 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed994a9-651e-4707-bc29-5af66ee4b7bc service placement] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:12.641200 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed994a9-651e-4707-bc29-5af66ee4b7bc service placement] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:12.701107 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231ed71-4777-42d9-a5d3-7f51ccb5c11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:12.701658 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231ed71-4777-42d9-a5d3-7f51ccb5c11d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:12.701936 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231ed71-4777-42d9-a5d3-7f51ccb5c11d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:12.702233 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231ed71-4777-42d9-a5d3-7f51ccb5c11d admin admin] PATH_INFO: `/v3/projects/0c5f5927d8cf48699c0691b46f9c461f` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:12.703344 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231ed71-4777-42d9-a5d3-7f51ccb5c11d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c5f5927d8cf48699c0691b46f9c461f)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:12.705218 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231ed71-4777-42d9-a5d3-7f51ccb5c11d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:12.750741 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d231ed71-4777-42d9-a5d3-7f51ccb5c11d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c5f5927d8cf48699c0691b46f9c461f', 'request_id': 'req-d231ed71-4777-42d9-a5d3-7f51ccb5c11d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:12.756732 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d231ed71-4777-42d9-a5d3-7f51ccb5c11d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c5f5927d8cf48699c0691b46f9c461f', 'request_id': 'req-d231ed71-4777-42d9-a5d3-7f51ccb5c11d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:12.762272 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d231ed71-4777-42d9-a5d3-7f51ccb5c11d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c5f5927d8cf48699c0691b46f9c461f', 'request_id': 'req-d231ed71-4777-42d9-a5d3-7f51ccb5c11d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:12.774756 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d231ed71-4777-42d9-a5d3-7f51ccb5c11d admin admin] The token cache is being invalidate because project 0c5f5927d8cf48699c0691b46f9c461f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:45:12.775068 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d231ed71-4777-42d9-a5d3-7f51ccb5c11d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:12.775695 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d231ed71-4777-42d9-a5d3-7f51ccb5c11d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:12.777136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1411/2796] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:45:12 2022] DELETE /identity/v3/projects/0c5f5927d8cf48699c0691b46f9c461f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:12.781887 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9aa0a-f461-41fc-aa6a-a2052092e49d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:12.783574 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f1f9aa0a-f461-41fc-aa6a-a2052092e49d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:12.797462 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9aa0a-f461-41fc-aa6a-a2052092e49d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:12.853282 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31eff59-a470-4229-bc19-4b991ec0ed73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:12.853905 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31eff59-a470-4229-bc19-4b991ec0ed73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:12.854226 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31eff59-a470-4229-bc19-4b991ec0ed73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:12.854496 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31eff59-a470-4229-bc19-4b991ec0ed73 admin admin] PATH_INFO: `/v3/users/ca20f893b7a84121a4ed22499c34c85b` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:12.855651 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31eff59-a470-4229-bc19-4b991ec0ed73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca20f893b7a84121a4ed22499c34c85b)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:12.858221 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31eff59-a470-4229-bc19-4b991ec0ed73 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:12.879187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:12.904733 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d31eff59-a470-4229-bc19-4b991ec0ed73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca20f893b7a84121a4ed22499c34c85b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:12.912939 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d31eff59-a470-4229-bc19-4b991ec0ed73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca20f893b7a84121a4ed22499c34c85b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:12.913609 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d31eff59-a470-4229-bc19-4b991ec0ed73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca20f893b7a84121a4ed22499c34c85b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:12.921481 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d31eff59-a470-4229-bc19-4b991ec0ed73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca20f893b7a84121a4ed22499c34c85b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:12.933369 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d31eff59-a470-4229-bc19-4b991ec0ed73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca20f893b7a84121a4ed22499c34c85b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:12.939316 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1386/2797] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:45:12 2022] DELETE /identity/v3/users/ca20f893b7a84121a4ed22499c34c85b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:13.110176 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231ed71-4777-42d9-a5d3-7f51ccb5c11d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:13.111711 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d231ed71-4777-42d9-a5d3-7f51ccb5c11d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:13.125042 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231ed71-4777-42d9-a5d3-7f51ccb5c11d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:13.149237 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:13.194629 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc8357a-335b-4a54-94fc-022acf200bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:13.195298 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8357a-335b-4a54-94fc-022acf200bde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:13.195782 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8357a-335b-4a54-94fc-022acf200bde admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:13.196056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8357a-335b-4a54-94fc-022acf200bde admin admin] PATH_INFO: `/v3/projects/2314b005429e473397437b879deaef43` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:13.197873 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc8357a-335b-4a54-94fc-022acf200bde admin admin] RBAC: Authorizing `identity:delete_project(project_id=2314b005429e473397437b879deaef43)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:13.199820 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc8357a-335b-4a54-94fc-022acf200bde admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:13.247575 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6bc8357a-335b-4a54-94fc-022acf200bde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2314b005429e473397437b879deaef43', 'request_id': 'req-6bc8357a-335b-4a54-94fc-022acf200bde'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:13.254095 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6bc8357a-335b-4a54-94fc-022acf200bde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2314b005429e473397437b879deaef43', 'request_id': 'req-6bc8357a-335b-4a54-94fc-022acf200bde'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:13.259593 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6bc8357a-335b-4a54-94fc-022acf200bde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2314b005429e473397437b879deaef43', 'request_id': 'req-6bc8357a-335b-4a54-94fc-022acf200bde'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:13.273086 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6bc8357a-335b-4a54-94fc-022acf200bde admin admin] The token cache is being invalidate because project 2314b005429e473397437b879deaef43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:45:13.273408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6bc8357a-335b-4a54-94fc-022acf200bde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:13.274091 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6bc8357a-335b-4a54-94fc-022acf200bde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:13.275664 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1412/2798] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:45:13 2022] DELETE /identity/v3/projects/2314b005429e473397437b879deaef43 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:13.281317 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0d65d-3750-4fb4-aead-8d12d26a922a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:13.281611 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0d65d-3750-4fb4-aead-8d12d26a922a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:13.281892 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0d65d-3750-4fb4-aead-8d12d26a922a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:13.302530 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5de0d65d-3750-4fb4-aead-8d12d26a922a None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:13.303754 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5de0d65d-3750-4fb4-aead-8d12d26a922a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:13.367654 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1387/2799] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:45:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:13.376045 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d11c4-2d8f-44d6-8017-5a831f226555 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:13.376504 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d11c4-2d8f-44d6-8017-5a831f226555 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:13.376834 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d11c4-2d8f-44d6-8017-5a831f226555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:13.397881 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-585d11c4-2d8f-44d6-8017-5a831f226555 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:13.399122 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-585d11c4-2d8f-44d6-8017-5a831f226555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:13.407573 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1413/2800] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:45:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:13.411817 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0d65d-3750-4fb4-aead-8d12d26a922a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:13.423168 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0d65d-3750-4fb4-aead-8d12d26a922a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:13.427720 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ceced2-577c-4c9f-a2a4-dc3da6f3ae34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:13.428325 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ceced2-577c-4c9f-a2a4-dc3da6f3ae34 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:13.428618 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ceced2-577c-4c9f-a2a4-dc3da6f3ae34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:13.428884 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ceced2-577c-4c9f-a2a4-dc3da6f3ae34 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:13.429593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ceced2-577c-4c9f-a2a4-dc3da6f3ae34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:45:13.429897 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ceced2-577c-4c9f-a2a4-dc3da6f3ae34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:13.431545 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ceced2-577c-4c9f-a2a4-dc3da6f3ae34 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:13.446495 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1388/2801] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:45:13.449358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d11c4-2d8f-44d6-8017-5a831f226555 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:13.460278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d11c4-2d8f-44d6-8017-5a831f226555 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:13.465387 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9aafca-5609-4e4c-86d6-c5918c75cec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:13.466804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9aafca-5609-4e4c-86d6-c5918c75cec6 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:13.467178 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9aafca-5609-4e4c-86d6-c5918c75cec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:13.467546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9aafca-5609-4e4c-86d6-c5918c75cec6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:13.468292 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9aafca-5609-4e4c-86d6-c5918c75cec6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:13.470358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9aafca-5609-4e4c-86d6-c5918c75cec6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:13.508151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1414/2802] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:45:13 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:13.511068 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ceced2-577c-4c9f-a2a4-dc3da6f3ae34 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:13.522152 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ceced2-577c-4c9f-a2a4-dc3da6f3ae34 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:13.586826 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873acef-748c-4c60-a26a-3b80895a8014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:13.588871 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873acef-748c-4c60-a26a-3b80895a8014 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:13.589169 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873acef-748c-4c60-a26a-3b80895a8014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:13.589454 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873acef-748c-4c60-a26a-3b80895a8014 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:13.590103 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5873acef-748c-4c60-a26a-3b80895a8014 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:13.592788 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5873acef-748c-4c60-a26a-3b80895a8014 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:13.614988 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1389/2803] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:45:13 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:13.618935 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9aafca-5609-4e4c-86d6-c5918c75cec6 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:13.629461 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9aafca-5609-4e4c-86d6-c5918c75cec6 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:13.634997 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96760889-d3f9-4056-a5a6-08f741981277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:13.635604 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96760889-d3f9-4056-a5a6-08f741981277 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:13.636013 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96760889-d3f9-4056-a5a6-08f741981277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:13.636300 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96760889-d3f9-4056-a5a6-08f741981277 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:13.636950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96760889-d3f9-4056-a5a6-08f741981277 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:13.638712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96760889-d3f9-4056-a5a6-08f741981277 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:13.649823 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1415/2804] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:45:13.654877 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873acef-748c-4c60-a26a-3b80895a8014 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:13.664999 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873acef-748c-4c60-a26a-3b80895a8014 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:13.670788 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e4293-4113-461a-8b2d-1a99a5d97a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:13.671388 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e4293-4113-461a-8b2d-1a99a5d97a25 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:13.671671 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e4293-4113-461a-8b2d-1a99a5d97a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:13.671946 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e4293-4113-461a-8b2d-1a99a5d97a25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:13.672592 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80e4293-4113-461a-8b2d-1a99a5d97a25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:13.674187 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80e4293-4113-461a-8b2d-1a99a5d97a25 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:13.685351 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1390/2805] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:13.690358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96760889-d3f9-4056-a5a6-08f741981277 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:13.701813 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96760889-d3f9-4056-a5a6-08f741981277 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:13.707146 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9466c0-d16d-47d7-a314-59dd3c0faf2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:13.707803 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9466c0-d16d-47d7-a314-59dd3c0faf2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:13.708082 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9466c0-d16d-47d7-a314-59dd3c0faf2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:13.708355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9466c0-d16d-47d7-a314-59dd3c0faf2e admin admin] PATH_INFO: `/v3/projects/19d165b4394343ff80b0da6faf2034bc/users/9301b98e87274de2a41eccca14819ebe/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:13.724540 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9466c0-d16d-47d7-a314-59dd3c0faf2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=19d165b4394343ff80b0da6faf2034bc, user_id=9301b98e87274de2a41eccca14819ebe, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:13.727236 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9466c0-d16d-47d7-a314-59dd3c0faf2e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:13.743345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1416/2806] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:13 2022] PUT /identity/v3/projects/19d165b4394343ff80b0da6faf2034bc/users/9301b98e87274de2a41eccca14819ebe/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:13.811202 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152378c7-3e2e-4466-90aa-8c97aed0cfd0 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:13.811488 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152378c7-3e2e-4466-90aa-8c97aed0cfd0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:13.811763 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152378c7-3e2e-4466-90aa-8c97aed0cfd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:13.820012 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:13.844393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-152378c7-3e2e-4466-90aa-8c97aed0cfd0 None None] MFA Rules not processed for user `9301b98e87274de2a41eccca14819ebe`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:13.845650 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-152378c7-3e2e-4466-90aa-8c97aed0cfd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:13.917460 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1391/2807] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:45:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:13.922006 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9466c0-d16d-47d7-a314-59dd3c0faf2e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:13.971753 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:13.987724 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9466c0-d16d-47d7-a314-59dd3c0faf2e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:13.992763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9299ee-301d-4dc4-bfe8-4f9c76dfbd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:13.993306 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9299ee-301d-4dc4-bfe8-4f9c76dfbd3b admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:13.993589 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9299ee-301d-4dc4-bfe8-4f9c76dfbd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:13.993862 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9299ee-301d-4dc4-bfe8-4f9c76dfbd3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:13.994550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9299ee-301d-4dc4-bfe8-4f9c76dfbd3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:13.996499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9299ee-301d-4dc4-bfe8-4f9c76dfbd3b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:14.024091 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1417/2808] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:45:13 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:45:14.028165 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152378c7-3e2e-4466-90aa-8c97aed0cfd0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:14.039670 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152378c7-3e2e-4466-90aa-8c97aed0cfd0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:14.098695 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b7551-a5dd-4292-8828-9cbb597f9018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:14.099306 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b7551-a5dd-4292-8828-9cbb597f9018 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:14.099587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b7551-a5dd-4292-8828-9cbb597f9018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:14.099862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b7551-a5dd-4292-8828-9cbb597f9018 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:14.100492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b7551-a5dd-4292-8828-9cbb597f9018 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:14.102458 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b7551-a5dd-4292-8828-9cbb597f9018 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:14.126297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1392/2809] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:45:14 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:14.130454 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9299ee-301d-4dc4-bfe8-4f9c76dfbd3b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:14.141318 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9299ee-301d-4dc4-bfe8-4f9c76dfbd3b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:14.146634 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1142c5-5652-45ac-82ef-2f9883fb9033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:14.147232 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1142c5-5652-45ac-82ef-2f9883fb9033 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:14.147510 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1142c5-5652-45ac-82ef-2f9883fb9033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:14.147786 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1142c5-5652-45ac-82ef-2f9883fb9033 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:14.148444 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1142c5-5652-45ac-82ef-2f9883fb9033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:14.150048 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1142c5-5652-45ac-82ef-2f9883fb9033 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:14.165007 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1418/2810] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:14.169509 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b7551-a5dd-4292-8828-9cbb597f9018 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:14.183565 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b7551-a5dd-4292-8828-9cbb597f9018 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:14.188560 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535732d0-454b-4cdd-add3-2f85c6eb1492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:14.189218 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535732d0-454b-4cdd-add3-2f85c6eb1492 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:14.189503 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535732d0-454b-4cdd-add3-2f85c6eb1492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:14.189781 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535732d0-454b-4cdd-add3-2f85c6eb1492 admin admin] PATH_INFO: `/v3/projects/add3155508794ecdab28201746b5bf25/users/07e214f21f384e30a2b5a95057aba0b2/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:14.206315 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535732d0-454b-4cdd-add3-2f85c6eb1492 admin admin] RBAC: Authorizing `identity:create_grant(project_id=add3155508794ecdab28201746b5bf25, user_id=07e214f21f384e30a2b5a95057aba0b2, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:14.209007 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535732d0-454b-4cdd-add3-2f85c6eb1492 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:14.223173 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1393/2811] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:14 2022] PUT /identity/v3/projects/add3155508794ecdab28201746b5bf25/users/07e214f21f384e30a2b5a95057aba0b2/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:14.227342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1142c5-5652-45ac-82ef-2f9883fb9033 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:14.238110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1142c5-5652-45ac-82ef-2f9883fb9033 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:14.297097 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40576f7e-f7f9-40a2-bee0-934f4a777fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:14.297632 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40576f7e-f7f9-40a2-bee0-934f4a777fb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:14.297941 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40576f7e-f7f9-40a2-bee0-934f4a777fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:14.298241 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40576f7e-f7f9-40a2-bee0-934f4a777fb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:14.298900 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40576f7e-f7f9-40a2-bee0-934f4a777fb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:14.300637 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40576f7e-f7f9-40a2-bee0-934f4a777fb3 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:14.311432 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1419/2812] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:14.315830 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535732d0-454b-4cdd-add3-2f85c6eb1492 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:14.327422 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535732d0-454b-4cdd-add3-2f85c6eb1492 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:14.332692 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff76ad-ca91-4f3c-9ac3-37d91d3ceb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:14.333365 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff76ad-ca91-4f3c-9ac3-37d91d3ceb98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:14.333649 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff76ad-ca91-4f3c-9ac3-37d91d3ceb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:14.333927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff76ad-ca91-4f3c-9ac3-37d91d3ceb98 admin admin] PATH_INFO: `/v3/projects/add3155508794ecdab28201746b5bf25/users/07e214f21f384e30a2b5a95057aba0b2/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:14.335923 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff76ad-ca91-4f3c-9ac3-37d91d3ceb98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=add3155508794ecdab28201746b5bf25, user_id=07e214f21f384e30a2b5a95057aba0b2, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:14.338602 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff76ad-ca91-4f3c-9ac3-37d91d3ceb98 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:14.349235 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1394/2813] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:14 2022] PUT /identity/v3/projects/add3155508794ecdab28201746b5bf25/users/07e214f21f384e30a2b5a95057aba0b2/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:14.359836 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08383d81-905b-4f93-9591-db2b26fea88b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:14.360242 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08383d81-905b-4f93-9591-db2b26fea88b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:14.360586 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08383d81-905b-4f93-9591-db2b26fea88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:14.391874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-08383d81-905b-4f93-9591-db2b26fea88b None None] MFA Rules not processed for user `07e214f21f384e30a2b5a95057aba0b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:14.393089 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-08383d81-905b-4f93-9591-db2b26fea88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:14.461359 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1420/2814] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:45:14 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:14.472585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff76ad-ca91-4f3c-9ac3-37d91d3ceb98 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:14.474341 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-62ff76ad-ca91-4f3c-9ac3-37d91d3ceb98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:14.528580 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:14.537748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff76ad-ca91-4f3c-9ac3-37d91d3ceb98 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:14.544093 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d150a-67ff-4c3f-8abc-9f3b757f4856 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:14.544706 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d150a-67ff-4c3f-8abc-9f3b757f4856 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:14.544990 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d150a-67ff-4c3f-8abc-9f3b757f4856 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:14.545265 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d150a-67ff-4c3f-8abc-9f3b757f4856 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:14.610042 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d150a-67ff-4c3f-8abc-9f3b757f4856 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:14.611508 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d150a-67ff-4c3f-8abc-9f3b757f4856 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:14.620278 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1395/2815] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:18.881530 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08383d81-905b-4f93-9591-db2b26fea88b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:18.883045 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-08383d81-905b-4f93-9591-db2b26fea88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:18.896138 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08383d81-905b-4f93-9591-db2b26fea88b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:18.899661 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7670479-bcdc-41c0-b93f-2c0fe0d62115 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:18.900253 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7670479-bcdc-41c0-b93f-2c0fe0d62115 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:18.900526 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7670479-bcdc-41c0-b93f-2c0fe0d62115 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:18.900794 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7670479-bcdc-41c0-b93f-2c0fe0d62115 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:18.909452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7670479-bcdc-41c0-b93f-2c0fe0d62115 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:18.910883 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7670479-bcdc-41c0-b93f-2c0fe0d62115 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:18.920375 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1421/2816] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:18 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:23.836745 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d150a-67ff-4c3f-8abc-9f3b757f4856 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:23.905574 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d150a-67ff-4c3f-8abc-9f3b757f4856 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:23.911360 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef7feb-abcc-4fa4-97db-1de22d64791c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:23.911995 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef7feb-abcc-4fa4-97db-1de22d64791c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:23.912394 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef7feb-abcc-4fa4-97db-1de22d64791c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:23.912677 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef7feb-abcc-4fa4-97db-1de22d64791c admin admin] PATH_INFO: `/v3/users/9301b98e87274de2a41eccca14819ebe` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:23.913856 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ef7feb-abcc-4fa4-97db-1de22d64791c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9301b98e87274de2a41eccca14819ebe)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:23.918437 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ef7feb-abcc-4fa4-97db-1de22d64791c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:23.965439 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-76ef7feb-abcc-4fa4-97db-1de22d64791c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9301b98e87274de2a41eccca14819ebe'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:23.973192 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-76ef7feb-abcc-4fa4-97db-1de22d64791c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9301b98e87274de2a41eccca14819ebe'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:23.973910 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-76ef7feb-abcc-4fa4-97db-1de22d64791c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9301b98e87274de2a41eccca14819ebe'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:23.981865 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-76ef7feb-abcc-4fa4-97db-1de22d64791c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9301b98e87274de2a41eccca14819ebe'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:23.996552 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-76ef7feb-abcc-4fa4-97db-1de22d64791c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9301b98e87274de2a41eccca14819ebe'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:24.002798 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1396/2817] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:45:23 2022] DELETE /identity/v3/users/9301b98e87274de2a41eccca14819ebe => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:24.007382 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58479463-dfcc-4454-a320-84f403dea9d2 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:24.007850 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58479463-dfcc-4454-a320-84f403dea9d2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:24.007954 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58479463-dfcc-4454-a320-84f403dea9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:24.026757 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-58479463-dfcc-4454-a320-84f403dea9d2 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:24.027889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-58479463-dfcc-4454-a320-84f403dea9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:24.055405 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:24.107981 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1422/2818] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:45:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:24.119153 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef7feb-abcc-4fa4-97db-1de22d64791c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:24.120676 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-76ef7feb-abcc-4fa4-97db-1de22d64791c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:24.152713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:24.203663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef7feb-abcc-4fa4-97db-1de22d64791c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:24.207483 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eae6ad-8554-4ffc-83d2-1722e1814570 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:24.208107 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eae6ad-8554-4ffc-83d2-1722e1814570 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:24.208391 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eae6ad-8554-4ffc-83d2-1722e1814570 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:24.208668 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eae6ad-8554-4ffc-83d2-1722e1814570 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:24.221764 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eae6ad-8554-4ffc-83d2-1722e1814570 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:24.224370 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eae6ad-8554-4ffc-83d2-1722e1814570 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:24.235825 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1397/2819] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:24.560707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58479463-dfcc-4454-a320-84f403dea9d2 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:24.562205 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-58479463-dfcc-4454-a320-84f403dea9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:24.574960 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58479463-dfcc-4454-a320-84f403dea9d2 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:24.579854 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e625c0-621b-46f9-90b0-0c2661ebbc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:24.580494 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e625c0-621b-46f9-90b0-0c2661ebbc88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:24.580924 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e625c0-621b-46f9-90b0-0c2661ebbc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:24.581220 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e625c0-621b-46f9-90b0-0c2661ebbc88 admin admin] PATH_INFO: `/v3/projects/19d165b4394343ff80b0da6faf2034bc` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:24.582664 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e625c0-621b-46f9-90b0-0c2661ebbc88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19d165b4394343ff80b0da6faf2034bc)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:24.585709 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e625c0-621b-46f9-90b0-0c2661ebbc88 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:24.635308 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f6e625c0-621b-46f9-90b0-0c2661ebbc88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19d165b4394343ff80b0da6faf2034bc', 'request_id': 'req-f6e625c0-621b-46f9-90b0-0c2661ebbc88'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:24.642090 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f6e625c0-621b-46f9-90b0-0c2661ebbc88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19d165b4394343ff80b0da6faf2034bc', 'request_id': 'req-f6e625c0-621b-46f9-90b0-0c2661ebbc88'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:24.648550 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f6e625c0-621b-46f9-90b0-0c2661ebbc88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19d165b4394343ff80b0da6faf2034bc', 'request_id': 'req-f6e625c0-621b-46f9-90b0-0c2661ebbc88'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:24.663055 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f6e625c0-621b-46f9-90b0-0c2661ebbc88 admin admin] The token cache is being invalidate because project 19d165b4394343ff80b0da6faf2034bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:45:24.663346 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f6e625c0-621b-46f9-90b0-0c2661ebbc88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:24.663983 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f6e625c0-621b-46f9-90b0-0c2661ebbc88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:24.665578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1423/2820] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:45:24 2022] DELETE /identity/v3/projects/19d165b4394343ff80b0da6faf2034bc => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:24.669857 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eae6ad-8554-4ffc-83d2-1722e1814570 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:24.680847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eae6ad-8554-4ffc-83d2-1722e1814570 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:24.765448 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f03be-5d04-4ca1-915d-cccf0d1791c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:24.765448 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f03be-5d04-4ca1-915d-cccf0d1791c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:24.765448 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f03be-5d04-4ca1-915d-cccf0d1791c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:24.765448 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f03be-5d04-4ca1-915d-cccf0d1791c1 admin admin] PATH_INFO: `/v3/users/07e214f21f384e30a2b5a95057aba0b2` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:24.768527 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f03be-5d04-4ca1-915d-cccf0d1791c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07e214f21f384e30a2b5a95057aba0b2)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:24.770613 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f03be-5d04-4ca1-915d-cccf0d1791c1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:24.819414 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-fd4f03be-5d04-4ca1-915d-cccf0d1791c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07e214f21f384e30a2b5a95057aba0b2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:24.826386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-fd4f03be-5d04-4ca1-915d-cccf0d1791c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07e214f21f384e30a2b5a95057aba0b2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:24.827089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-fd4f03be-5d04-4ca1-915d-cccf0d1791c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07e214f21f384e30a2b5a95057aba0b2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:24.834619 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-fd4f03be-5d04-4ca1-915d-cccf0d1791c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07e214f21f384e30a2b5a95057aba0b2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:24.846859 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-fd4f03be-5d04-4ca1-915d-cccf0d1791c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07e214f21f384e30a2b5a95057aba0b2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:24.852757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1398/2821] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:45:24 2022] DELETE /identity/v3/users/07e214f21f384e30a2b5a95057aba0b2 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:25.138547 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e625c0-621b-46f9-90b0-0c2661ebbc88 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:25.140175 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f6e625c0-621b-46f9-90b0-0c2661ebbc88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:25.155420 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e625c0-621b-46f9-90b0-0c2661ebbc88 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:25.228269 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4778036-db47-4650-899f-af4617a882cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:25.228759 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4778036-db47-4650-899f-af4617a882cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:25.229042 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4778036-db47-4650-899f-af4617a882cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:25.229317 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4778036-db47-4650-899f-af4617a882cf admin admin] PATH_INFO: `/v3/projects/add3155508794ecdab28201746b5bf25` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:25.230501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4778036-db47-4650-899f-af4617a882cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=add3155508794ecdab28201746b5bf25)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:25.232442 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4778036-db47-4650-899f-af4617a882cf admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:25.278717 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f4778036-db47-4650-899f-af4617a882cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'add3155508794ecdab28201746b5bf25', 'request_id': 'req-f4778036-db47-4650-899f-af4617a882cf'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:25.285361 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f4778036-db47-4650-899f-af4617a882cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'add3155508794ecdab28201746b5bf25', 'request_id': 'req-f4778036-db47-4650-899f-af4617a882cf'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:25.286588 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:25.293104 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f4778036-db47-4650-899f-af4617a882cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'add3155508794ecdab28201746b5bf25', 'request_id': 'req-f4778036-db47-4650-899f-af4617a882cf'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:25.309787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f4778036-db47-4650-899f-af4617a882cf admin admin] The token cache is being invalidate because project add3155508794ecdab28201746b5bf25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:45:25.310113 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f4778036-db47-4650-899f-af4617a882cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:25.310780 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f4778036-db47-4650-899f-af4617a882cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:25.312593 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1424/2822] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:45:25 2022] DELETE /identity/v3/projects/add3155508794ecdab28201746b5bf25 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:25.318789 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9bbeee-66ce-47af-8b06-c4f1de6be4c7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:25.319147 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9bbeee-66ce-47af-8b06-c4f1de6be4c7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:25.319493 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9bbeee-66ce-47af-8b06-c4f1de6be4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:25.342986 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9e9bbeee-66ce-47af-8b06-c4f1de6be4c7 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:25.344376 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9e9bbeee-66ce-47af-8b06-c4f1de6be4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:25.386577 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:25.415092 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1399/2823] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:45:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:25.423492 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba332a2-54d3-4176-bf89-f719675eb3a7 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:25.423780 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba332a2-54d3-4176-bf89-f719675eb3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:25.424054 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba332a2-54d3-4176-bf89-f719675eb3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:25.444132 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-9ba332a2-54d3-4176-bf89-f719675eb3a7 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:25.445342 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9ba332a2-54d3-4176-bf89-f719675eb3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:25.453561 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1425/2824] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:45:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:25.458492 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9bbeee-66ce-47af-8b06-c4f1de6be4c7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:25.468071 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9bbeee-66ce-47af-8b06-c4f1de6be4c7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:25.473495 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db74e64-c82a-4623-8a66-70c6e2540396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:25.474108 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db74e64-c82a-4623-8a66-70c6e2540396 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:25.474520 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db74e64-c82a-4623-8a66-70c6e2540396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:25.474778 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db74e64-c82a-4623-8a66-70c6e2540396 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:25.475431 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db74e64-c82a-4623-8a66-70c6e2540396 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:45:25.475735 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db74e64-c82a-4623-8a66-70c6e2540396 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:25.477571 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db74e64-c82a-4623-8a66-70c6e2540396 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:25.489540 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1400/2825] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:45:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:45:25.493770 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba332a2-54d3-4176-bf89-f719675eb3a7 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:25.503932 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba332a2-54d3-4176-bf89-f719675eb3a7 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:25.508826 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d668f-5601-48f3-929b-bd8110230042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:25.509438 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d668f-5601-48f3-929b-bd8110230042 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:25.509744 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d668f-5601-48f3-929b-bd8110230042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:25.510015 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d668f-5601-48f3-929b-bd8110230042 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:25.510697 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d668f-5601-48f3-929b-bd8110230042 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:25.512590 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d668f-5601-48f3-929b-bd8110230042 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:25.520731 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db74e64-c82a-4623-8a66-70c6e2540396 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:25.522435 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3db74e64-c82a-4623-8a66-70c6e2540396 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:25.544343 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1426/2826] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:45:25 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:25.548222 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d668f-5601-48f3-929b-bd8110230042 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:25.581362 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db74e64-c82a-4623-8a66-70c6e2540396 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:25.618901 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d668f-5601-48f3-929b-bd8110230042 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:25.624145 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4ca93-9c85-401f-8eef-18062686eb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:25.624823 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4ca93-9c85-401f-8eef-18062686eb81 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:25.625109 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4ca93-9c85-401f-8eef-18062686eb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:25.625502 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4ca93-9c85-401f-8eef-18062686eb81 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:25.626235 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b4ca93-9c85-401f-8eef-18062686eb81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:25.628303 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b4ca93-9c85-401f-8eef-18062686eb81 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:25.632203 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478414e3-c2f5-4be8-875a-7a81a67674b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:25.632854 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478414e3-c2f5-4be8-875a-7a81a67674b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:25.633106 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478414e3-c2f5-4be8-875a-7a81a67674b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:25.633377 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478414e3-c2f5-4be8-875a-7a81a67674b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:25.635346 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-478414e3-c2f5-4be8-875a-7a81a67674b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:25.645516 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478414e3-c2f5-4be8-875a-7a81a67674b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:25.646984 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478414e3-c2f5-4be8-875a-7a81a67674b8 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:25.650034 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1427/2827] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:45:25 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:45:25.654479 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4ca93-9c85-401f-8eef-18062686eb81 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:25.661883 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1401/2828] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:45:25.665224 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4ca93-9c85-401f-8eef-18062686eb81 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:25.670106 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a558ff-832a-43a2-9283-732303ae40e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:25.670745 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a558ff-832a-43a2-9283-732303ae40e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:25.671023 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a558ff-832a-43a2-9283-732303ae40e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:25.671321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a558ff-832a-43a2-9283-732303ae40e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:25.671969 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a558ff-832a-43a2-9283-732303ae40e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:25.673873 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a558ff-832a-43a2-9283-732303ae40e6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:25.685950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1428/2829] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:45:25.690373 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478414e3-c2f5-4be8-875a-7a81a67674b8 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:25.701971 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478414e3-c2f5-4be8-875a-7a81a67674b8 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:25.707956 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed2ab7b-5392-4926-9e0d-42aa9802b5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:25.708644 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2ab7b-5392-4926-9e0d-42aa9802b5a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:25.709032 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2ab7b-5392-4926-9e0d-42aa9802b5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:25.709500 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2ab7b-5392-4926-9e0d-42aa9802b5a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:25.710315 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed2ab7b-5392-4926-9e0d-42aa9802b5a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:25.712420 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed2ab7b-5392-4926-9e0d-42aa9802b5a9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:25.727655 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1402/2830] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:25.728954 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a558ff-832a-43a2-9283-732303ae40e6 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:25.740078 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a558ff-832a-43a2-9283-732303ae40e6 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:25.748071 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2aefd-272b-4239-a532-3e936cae724d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:25.748748 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2aefd-272b-4239-a532-3e936cae724d admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:25.750492 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2aefd-272b-4239-a532-3e936cae724d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:25.751062 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2aefd-272b-4239-a532-3e936cae724d admin admin] PATH_INFO: `/v3/projects/98f9be55e17a4a80a9b6cbf41a5f173c/users/56995959809642aa862e16ca23303c07/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:25.778570 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da2aefd-272b-4239-a532-3e936cae724d admin admin] RBAC: Authorizing `identity:create_grant(project_id=98f9be55e17a4a80a9b6cbf41a5f173c, user_id=56995959809642aa862e16ca23303c07, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:25.781377 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da2aefd-272b-4239-a532-3e936cae724d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:25.816874 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1429/2831] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:25 2022] PUT /identity/v3/projects/98f9be55e17a4a80a9b6cbf41a5f173c/users/56995959809642aa862e16ca23303c07/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:25.819081 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40eb4a-ec8c-4882-a873-5bbbb5dcf2a7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:25.819379 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40eb4a-ec8c-4882-a873-5bbbb5dcf2a7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:25.819659 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40eb4a-ec8c-4882-a873-5bbbb5dcf2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:25.849835 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-aa40eb4a-ec8c-4882-a873-5bbbb5dcf2a7 None None] MFA Rules not processed for user `56995959809642aa862e16ca23303c07`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:25.851080 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-aa40eb4a-ec8c-4882-a873-5bbbb5dcf2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:25.923852 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1403/2832] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:45:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:25.928518 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2aefd-272b-4239-a532-3e936cae724d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:25.992177 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2aefd-272b-4239-a532-3e936cae724d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:25.996866 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c28517-76cc-483f-88f1-bd17deb4af1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:25.997499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c28517-76cc-483f-88f1-bd17deb4af1b admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:25.997917 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c28517-76cc-483f-88f1-bd17deb4af1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:25.998218 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c28517-76cc-483f-88f1-bd17deb4af1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:25.998944 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c28517-76cc-483f-88f1-bd17deb4af1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:26.000845 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c28517-76cc-483f-88f1-bd17deb4af1b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:26.015451 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=3,keystone:SELECT=91,keystone:INSERT=6 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:26.032199 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1430/2833] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:45:25 2022] POST /identity/v3/projects => generated 369 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:26.036256 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40eb4a-ec8c-4882-a873-5bbbb5dcf2a7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:26.052477 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40eb4a-ec8c-4882-a873-5bbbb5dcf2a7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:26.056829 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:26.115273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36779688-a812-452d-b7a5-bbedd3c170d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:26.115961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36779688-a812-452d-b7a5-bbedd3c170d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:26.116249 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36779688-a812-452d-b7a5-bbedd3c170d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:26.116528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36779688-a812-452d-b7a5-bbedd3c170d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:26.117172 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36779688-a812-452d-b7a5-bbedd3c170d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:26.119134 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36779688-a812-452d-b7a5-bbedd3c170d8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:26.142964 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1404/2834] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:45:26 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:45:26.147000 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c28517-76cc-483f-88f1-bd17deb4af1b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:26.158405 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c28517-76cc-483f-88f1-bd17deb4af1b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:26.163324 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44bcf68-8caf-49e4-a81a-d6ec0d0e6eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:26.164044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bcf68-8caf-49e4-a81a-d6ec0d0e6eba admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:26.164380 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bcf68-8caf-49e4-a81a-d6ec0d0e6eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:26.164747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bcf68-8caf-49e4-a81a-d6ec0d0e6eba admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:26.165465 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44bcf68-8caf-49e4-a81a-d6ec0d0e6eba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:26.167620 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44bcf68-8caf-49e4-a81a-d6ec0d0e6eba admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:26.186907 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1431/2835] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:26.188754 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36779688-a812-452d-b7a5-bbedd3c170d8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:26.200166 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36779688-a812-452d-b7a5-bbedd3c170d8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:26.205804 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4fa52-cfd0-474c-a5ab-aa2c92382be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:26.206574 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4fa52-cfd0-474c-a5ab-aa2c92382be2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:26.206919 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4fa52-cfd0-474c-a5ab-aa2c92382be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:26.207262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4fa52-cfd0-474c-a5ab-aa2c92382be2 admin admin] PATH_INFO: `/v3/projects/b6742ebad0bb4009be0033996b773226/users/16307afe10c34557bf91a262e9f5aff3/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:26.236172 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4fa52-cfd0-474c-a5ab-aa2c92382be2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6742ebad0bb4009be0033996b773226, user_id=16307afe10c34557bf91a262e9f5aff3, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:26.239103 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4fa52-cfd0-474c-a5ab-aa2c92382be2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:26.253858 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1405/2836] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:26 2022] PUT /identity/v3/projects/b6742ebad0bb4009be0033996b773226/users/16307afe10c34557bf91a262e9f5aff3/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:26.265787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44bcf68-8caf-49e4-a81a-d6ec0d0e6eba admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:26.277052 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44bcf68-8caf-49e4-a81a-d6ec0d0e6eba admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:26.337925 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727c202-842d-4367-8d2a-e325cad3f90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:26.338563 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727c202-842d-4367-8d2a-e325cad3f90c admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:26.338855 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727c202-842d-4367-8d2a-e325cad3f90c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:26.339128 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727c202-842d-4367-8d2a-e325cad3f90c admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:26.339809 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0727c202-842d-4367-8d2a-e325cad3f90c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:26.341401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0727c202-842d-4367-8d2a-e325cad3f90c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:26.352896 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1432/2837] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:26.357203 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4fa52-cfd0-474c-a5ab-aa2c92382be2 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:26.368067 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4fa52-cfd0-474c-a5ab-aa2c92382be2 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:26.372857 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f7475-a4fb-4e1c-8c56-ecd0b190ae35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:26.373507 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f7475-a4fb-4e1c-8c56-ecd0b190ae35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:26.373792 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f7475-a4fb-4e1c-8c56-ecd0b190ae35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:26.374069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f7475-a4fb-4e1c-8c56-ecd0b190ae35 admin admin] PATH_INFO: `/v3/projects/b6742ebad0bb4009be0033996b773226/users/16307afe10c34557bf91a262e9f5aff3/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:26.376668 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f7475-a4fb-4e1c-8c56-ecd0b190ae35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6742ebad0bb4009be0033996b773226, user_id=16307afe10c34557bf91a262e9f5aff3, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:26.379340 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f7475-a4fb-4e1c-8c56-ecd0b190ae35 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:26.390001 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1406/2838] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:26 2022] PUT /identity/v3/projects/b6742ebad0bb4009be0033996b773226/users/16307afe10c34557bf91a262e9f5aff3/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:45:26.401028 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0d9f6-d2d0-4ec0-9671-174f1f4bd4a8 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:26.401338 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0d9f6-d2d0-4ec0-9671-174f1f4bd4a8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:26.401612 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0d9f6-d2d0-4ec0-9671-174f1f4bd4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:26.430679 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ade0d9f6-d2d0-4ec0-9671-174f1f4bd4a8 None None] MFA Rules not processed for user `16307afe10c34557bf91a262e9f5aff3`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:26.431894 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ade0d9f6-d2d0-4ec0-9671-174f1f4bd4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:26.499585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1433/2839] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:45:26 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:26.514122 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f7475-a4fb-4e1c-8c56-ecd0b190ae35 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:26.516024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dc4f7475-a4fb-4e1c-8c56-ecd0b190ae35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:26.573700 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f7475-a4fb-4e1c-8c56-ecd0b190ae35 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:26.577280 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d922adb-e6ae-4bb2-87d6-c69d3ad2e9fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:26.577875 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d922adb-e6ae-4bb2-87d6-c69d3ad2e9fb service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:26.578154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d922adb-e6ae-4bb2-87d6-c69d3ad2e9fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:26.578458 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d922adb-e6ae-4bb2-87d6-c69d3ad2e9fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:26.632779 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d922adb-e6ae-4bb2-87d6-c69d3ad2e9fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:26.634403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d922adb-e6ae-4bb2-87d6-c69d3ad2e9fb service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:26.643324 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1407/2840] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:29.204641 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0d9f6-d2d0-4ec0-9671-174f1f4bd4a8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:29.269917 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0d9f6-d2d0-4ec0-9671-174f1f4bd4a8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:29.274501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480251c-04fd-4ffe-a194-5b543514b62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:29.275118 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480251c-04fd-4ffe-a194-5b543514b62b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:29.275396 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480251c-04fd-4ffe-a194-5b543514b62b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:29.275666 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480251c-04fd-4ffe-a194-5b543514b62b admin admin] PATH_INFO: `/v3/users/56995959809642aa862e16ca23303c07` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:29.276775 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480251c-04fd-4ffe-a194-5b543514b62b admin admin] RBAC: Authorizing `identity:delete_user(user_id=56995959809642aa862e16ca23303c07)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:29.278733 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480251c-04fd-4ffe-a194-5b543514b62b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:29.283249 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:29.325504 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c480251c-04fd-4ffe-a194-5b543514b62b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56995959809642aa862e16ca23303c07'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:29.332814 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c480251c-04fd-4ffe-a194-5b543514b62b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56995959809642aa862e16ca23303c07'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:29.333474 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c480251c-04fd-4ffe-a194-5b543514b62b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56995959809642aa862e16ca23303c07'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:29.334085 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c480251c-04fd-4ffe-a194-5b543514b62b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56995959809642aa862e16ca23303c07'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:29.345950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c480251c-04fd-4ffe-a194-5b543514b62b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56995959809642aa862e16ca23303c07'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:29.361160 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1434/2841] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:45:29 2022] DELETE /identity/v3/users/56995959809642aa862e16ca23303c07 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:29.365551 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da31596-b26a-4dc6-8570-520af844f782 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:29.366027 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da31596-b26a-4dc6-8570-520af844f782 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:29.366132 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da31596-b26a-4dc6-8570-520af844f782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:29.387997 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6da31596-b26a-4dc6-8570-520af844f782 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:29.389222 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6da31596-b26a-4dc6-8570-520af844f782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:29.402378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:29.464539 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1408/2842] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:45:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:29.475182 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480251c-04fd-4ffe-a194-5b543514b62b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:29.476650 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c480251c-04fd-4ffe-a194-5b543514b62b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:29.535014 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480251c-04fd-4ffe-a194-5b543514b62b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:29.538712 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e5242-f1a9-495a-ba12-6fa9b7dab4f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:29.539335 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e5242-f1a9-495a-ba12-6fa9b7dab4f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:29.539615 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e5242-f1a9-495a-ba12-6fa9b7dab4f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:29.539889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e5242-f1a9-495a-ba12-6fa9b7dab4f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:29.549594 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e5242-f1a9-495a-ba12-6fa9b7dab4f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:29.551048 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e5242-f1a9-495a-ba12-6fa9b7dab4f2 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:29.560281 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1435/2843] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:29.852695 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da31596-b26a-4dc6-8570-520af844f782 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:29.854234 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6da31596-b26a-4dc6-8570-520af844f782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:29.866271 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da31596-b26a-4dc6-8570-520af844f782 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:29.870841 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271045e-9056-495f-9f6c-d5ee82934b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:29.871475 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271045e-9056-495f-9f6c-d5ee82934b65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:29.871754 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271045e-9056-495f-9f6c-d5ee82934b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:29.872026 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271045e-9056-495f-9f6c-d5ee82934b65 admin admin] PATH_INFO: `/v3/projects/98f9be55e17a4a80a9b6cbf41a5f173c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:29.873239 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271045e-9056-495f-9f6c-d5ee82934b65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98f9be55e17a4a80a9b6cbf41a5f173c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:29.875083 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271045e-9056-495f-9f6c-d5ee82934b65 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:29.921710 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0271045e-9056-495f-9f6c-d5ee82934b65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98f9be55e17a4a80a9b6cbf41a5f173c', 'request_id': 'req-0271045e-9056-495f-9f6c-d5ee82934b65'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:29.927862 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0271045e-9056-495f-9f6c-d5ee82934b65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98f9be55e17a4a80a9b6cbf41a5f173c', 'request_id': 'req-0271045e-9056-495f-9f6c-d5ee82934b65'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:29.935371 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0271045e-9056-495f-9f6c-d5ee82934b65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98f9be55e17a4a80a9b6cbf41a5f173c', 'request_id': 'req-0271045e-9056-495f-9f6c-d5ee82934b65'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:29.946275 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0271045e-9056-495f-9f6c-d5ee82934b65 admin admin] The token cache is being invalidate because project 98f9be55e17a4a80a9b6cbf41a5f173c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:45:29.946568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0271045e-9056-495f-9f6c-d5ee82934b65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:29.947199 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0271045e-9056-495f-9f6c-d5ee82934b65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:29.948763 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1409/2844] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:45:29 2022] DELETE /identity/v3/projects/98f9be55e17a4a80a9b6cbf41a5f173c => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:29.952707 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e5242-f1a9-495a-ba12-6fa9b7dab4f2 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:29.963175 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e5242-f1a9-495a-ba12-6fa9b7dab4f2 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:30.018967 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635b916-b9a8-4a60-b299-e2703a14ba97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:30.019582 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635b916-b9a8-4a60-b299-e2703a14ba97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:30.019860 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635b916-b9a8-4a60-b299-e2703a14ba97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:30.020148 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635b916-b9a8-4a60-b299-e2703a14ba97 admin admin] PATH_INFO: `/v3/users/16307afe10c34557bf91a262e9f5aff3` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:30.021250 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1635b916-b9a8-4a60-b299-e2703a14ba97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16307afe10c34557bf91a262e9f5aff3)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:30.023207 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1635b916-b9a8-4a60-b299-e2703a14ba97 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:30.051943 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:30.071548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1635b916-b9a8-4a60-b299-e2703a14ba97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16307afe10c34557bf91a262e9f5aff3'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:30.078251 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1635b916-b9a8-4a60-b299-e2703a14ba97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16307afe10c34557bf91a262e9f5aff3'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:30.078910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1635b916-b9a8-4a60-b299-e2703a14ba97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16307afe10c34557bf91a262e9f5aff3'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:30.079537 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1635b916-b9a8-4a60-b299-e2703a14ba97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16307afe10c34557bf91a262e9f5aff3'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:30.091688 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1635b916-b9a8-4a60-b299-e2703a14ba97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16307afe10c34557bf91a262e9f5aff3'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:30.104572 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1436/2845] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:45:29 2022] DELETE /identity/v3/users/16307afe10c34557bf91a262e9f5aff3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:30.380018 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271045e-9056-495f-9f6c-d5ee82934b65 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:30.381590 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0271045e-9056-495f-9f6c-d5ee82934b65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:30.396049 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271045e-9056-495f-9f6c-d5ee82934b65 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:30.458246 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d6643-d0b5-4e96-85ef-a5bfb6520c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:30.458802 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d6643-d0b5-4e96-85ef-a5bfb6520c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:30.459114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d6643-d0b5-4e96-85ef-a5bfb6520c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:30.459392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d6643-d0b5-4e96-85ef-a5bfb6520c66 admin admin] PATH_INFO: `/v3/projects/b6742ebad0bb4009be0033996b773226` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:30.460511 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d6643-d0b5-4e96-85ef-a5bfb6520c66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6742ebad0bb4009be0033996b773226)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:30.462455 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d6643-d0b5-4e96-85ef-a5bfb6520c66 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:30.502605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:30.511442 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-781d6643-d0b5-4e96-85ef-a5bfb6520c66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6742ebad0bb4009be0033996b773226', 'request_id': 'req-781d6643-d0b5-4e96-85ef-a5bfb6520c66'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:30.517966 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-781d6643-d0b5-4e96-85ef-a5bfb6520c66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6742ebad0bb4009be0033996b773226', 'request_id': 'req-781d6643-d0b5-4e96-85ef-a5bfb6520c66'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:30.525204 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-781d6643-d0b5-4e96-85ef-a5bfb6520c66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6742ebad0bb4009be0033996b773226', 'request_id': 'req-781d6643-d0b5-4e96-85ef-a5bfb6520c66'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:30.535496 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-781d6643-d0b5-4e96-85ef-a5bfb6520c66 admin admin] The token cache is being invalidate because project b6742ebad0bb4009be0033996b773226 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:45:30.535785 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-781d6643-d0b5-4e96-85ef-a5bfb6520c66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:30.536397 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-781d6643-d0b5-4e96-85ef-a5bfb6520c66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:30.537855 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1410/2846] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:45:30 2022] DELETE /identity/v3/projects/b6742ebad0bb4009be0033996b773226 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:30.543216 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7709803-fdaa-48c9-8ee1-a43b659186a2 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:30.543714 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7709803-fdaa-48c9-8ee1-a43b659186a2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:30.543806 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7709803-fdaa-48c9-8ee1-a43b659186a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:30.563799 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a7709803-fdaa-48c9-8ee1-a43b659186a2 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:30.565605 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a7709803-fdaa-48c9-8ee1-a43b659186a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:30.627808 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1437/2847] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:30.636090 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c1390-3968-4a77-bea8-1f79af95081d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:30.636860 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c1390-3968-4a77-bea8-1f79af95081d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:30.636860 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c1390-3968-4a77-bea8-1f79af95081d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:30.656840 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-268c1390-3968-4a77-bea8-1f79af95081d None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:30.658595 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-268c1390-3968-4a77-bea8-1f79af95081d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:30.666553 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1411/2848] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:30.671054 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7709803-fdaa-48c9-8ee1-a43b659186a2 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:30.681135 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7709803-fdaa-48c9-8ee1-a43b659186a2 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:30.685690 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ce7f0-e8e9-4f96-933f-5d4a719f9e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:30.686318 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ce7f0-e8e9-4f96-933f-5d4a719f9e83 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:30.686591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ce7f0-e8e9-4f96-933f-5d4a719f9e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:30.686857 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ce7f0-e8e9-4f96-933f-5d4a719f9e83 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:30.687501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ce7f0-e8e9-4f96-933f-5d4a719f9e83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:45:30.687800 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ce7f0-e8e9-4f96-933f-5d4a719f9e83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:30.689903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ce7f0-e8e9-4f96-933f-5d4a719f9e83 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:30.701508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1438/2849] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:45:30.706424 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c1390-3968-4a77-bea8-1f79af95081d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:30.715770 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c1390-3968-4a77-bea8-1f79af95081d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:30.720376 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87502c98-40b0-46b4-8169-b24bb3738cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:30.720968 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87502c98-40b0-46b4-8169-b24bb3738cf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:30.721248 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87502c98-40b0-46b4-8169-b24bb3738cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:30.721519 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87502c98-40b0-46b4-8169-b24bb3738cf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:30.722184 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87502c98-40b0-46b4-8169-b24bb3738cf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:30.724167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87502c98-40b0-46b4-8169-b24bb3738cf9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:30.751122 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1412/2850] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:45:30 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:45:30.755795 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ce7f0-e8e9-4f96-933f-5d4a719f9e83 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:30.765511 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ce7f0-e8e9-4f96-933f-5d4a719f9e83 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:30.819407 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606ebb6-c0e8-4e3c-ab9e-7fb2830ef962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:30.820001 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606ebb6-c0e8-4e3c-ab9e-7fb2830ef962 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:30.820274 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606ebb6-c0e8-4e3c-ab9e-7fb2830ef962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:30.820544 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606ebb6-c0e8-4e3c-ab9e-7fb2830ef962 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:30.821159 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e606ebb6-c0e8-4e3c-ab9e-7fb2830ef962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:30.823064 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e606ebb6-c0e8-4e3c-ab9e-7fb2830ef962 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:30.844287 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1439/2851] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:45:30 2022] POST /identity/v3/users => generated 291 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:30.851163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87502c98-40b0-46b4-8169-b24bb3738cf9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:30.861959 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87502c98-40b0-46b4-8169-b24bb3738cf9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:30.866702 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9c123-5fa9-45d4-851d-9e9a0ffed2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:30.867292 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9c123-5fa9-45d4-851d-9e9a0ffed2d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:30.867573 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9c123-5fa9-45d4-851d-9e9a0ffed2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:30.867845 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9c123-5fa9-45d4-851d-9e9a0ffed2d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:30.868491 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e9c123-5fa9-45d4-851d-9e9a0ffed2d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:30.870064 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e9c123-5fa9-45d4-851d-9e9a0ffed2d1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:30.880125 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1413/2852] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:30.884178 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606ebb6-c0e8-4e3c-ab9e-7fb2830ef962 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:30.894910 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606ebb6-c0e8-4e3c-ab9e-7fb2830ef962 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:30.899583 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e037ae33-9531-420a-bb36-0418bdca4f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:30.900171 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037ae33-9531-420a-bb36-0418bdca4f16 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:30.900452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037ae33-9531-420a-bb36-0418bdca4f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:30.900725 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037ae33-9531-420a-bb36-0418bdca4f16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:30.901367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e037ae33-9531-420a-bb36-0418bdca4f16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:30.902972 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e037ae33-9531-420a-bb36-0418bdca4f16 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:30.913433 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1440/2853] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:30.918021 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9c123-5fa9-45d4-851d-9e9a0ffed2d1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:30.928174 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9c123-5fa9-45d4-851d-9e9a0ffed2d1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:30.932937 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db419956-aa41-4916-8226-e0ea088cad89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:30.933576 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db419956-aa41-4916-8226-e0ea088cad89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:30.933856 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db419956-aa41-4916-8226-e0ea088cad89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:30.934130 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db419956-aa41-4916-8226-e0ea088cad89 admin admin] PATH_INFO: `/v3/projects/829ea07fd4c24f87b6eeb96e8a6952a0/users/e8f521436e924bf6a4571d3aeda481cf/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:30.948903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db419956-aa41-4916-8226-e0ea088cad89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=829ea07fd4c24f87b6eeb96e8a6952a0, user_id=e8f521436e924bf6a4571d3aeda481cf, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:30.951562 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db419956-aa41-4916-8226-e0ea088cad89 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:30.965336 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1414/2854] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:30 2022] PUT /identity/v3/projects/829ea07fd4c24f87b6eeb96e8a6952a0/users/e8f521436e924bf6a4571d3aeda481cf/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:30.970928 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383bff2-a146-4e1e-96cd-ebcf256d05d1 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:30.971112 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383bff2-a146-4e1e-96cd-ebcf256d05d1 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:30.971381 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383bff2-a146-4e1e-96cd-ebcf256d05d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:30.990349 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-6383bff2-a146-4e1e-96cd-ebcf256d05d1 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:30.991527 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6383bff2-a146-4e1e-96cd-ebcf256d05d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:30.996664 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:31.051546 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1441/2855] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:31.061106 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db419956-aa41-4916-8226-e0ea088cad89 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:31.062914 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-db419956-aa41-4916-8226-e0ea088cad89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:31.119164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db419956-aa41-4916-8226-e0ea088cad89 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:31.122801 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9811ef-7d0f-4923-84d5-bf054e0cf2c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:31.123406 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9811ef-7d0f-4923-84d5-bf054e0cf2c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:31.123688 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9811ef-7d0f-4923-84d5-bf054e0cf2c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:31.123964 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9811ef-7d0f-4923-84d5-bf054e0cf2c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:31.132773 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9811ef-7d0f-4923-84d5-bf054e0cf2c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:31.134311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9811ef-7d0f-4923-84d5-bf054e0cf2c5 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:31.144116 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1415/2856] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:45:31.768548 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d321df-1e58-4a32-b38d-ecb4f1046693 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:31.769042 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d321df-1e58-4a32-b38d-ecb4f1046693 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:31.769147 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d321df-1e58-4a32-b38d-ecb4f1046693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:31.810681 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a7d321df-1e58-4a32-b38d-ecb4f1046693 None None] MFA Rules not processed for user `e8f521436e924bf6a4571d3aeda481cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:31.812043 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a7d321df-1e58-4a32-b38d-ecb4f1046693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:31.890269 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1442/2857] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:45:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:31.899409 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9811ef-7d0f-4923-84d5-bf054e0cf2c5 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:31.900872 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dd9811ef-7d0f-4923-84d5-bf054e0cf2c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:31.959724 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9811ef-7d0f-4923-84d5-bf054e0cf2c5 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:31.963284 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e39ca-15bf-4489-acae-805df6b7e9d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:31.963889 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e39ca-15bf-4489-acae-805df6b7e9d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:31.964164 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e39ca-15bf-4489-acae-805df6b7e9d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:31.964434 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e39ca-15bf-4489-acae-805df6b7e9d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:31.973052 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e39ca-15bf-4489-acae-805df6b7e9d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:31.974560 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e39ca-15bf-4489-acae-805df6b7e9d1 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:31.983151 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1416/2858] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:31 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:34.392096 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe37bc-e656-4410-84e9-0270cb656cd3 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:34.393175 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe37bc-e656-4410-84e9-0270cb656cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:34.393422 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe37bc-e656-4410-84e9-0270cb656cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:34.412279 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-8bbe37bc-e656-4410-84e9-0270cb656cd3 None None] MFA Rules not processed for user `e8f521436e924bf6a4571d3aeda481cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:34.413490 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8bbe37bc-e656-4410-84e9-0270cb656cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:34.420758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1443/2859] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:45:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:41.979193 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:43.947584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e39ca-15bf-4489-acae-805df6b7e9d1 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:43.949118 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5b2e39ca-15bf-4489-acae-805df6b7e9d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:43.961177 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e39ca-15bf-4489-acae-805df6b7e9d1 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:43.964706 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e815b0-c8e0-474b-b2a0-b2976900135d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:43.965297 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e815b0-c8e0-474b-b2a0-b2976900135d service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:43.965578 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e815b0-c8e0-474b-b2a0-b2976900135d service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:43.965847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e815b0-c8e0-474b-b2a0-b2976900135d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:43.967526 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b8e815b0-c8e0-474b-b2a0-b2976900135d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:44.054588 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e815b0-c8e0-474b-b2a0-b2976900135d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:44.056154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e815b0-c8e0-474b-b2a0-b2976900135d service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:44.071549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1417/2860] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:44.406824 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=57 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:46.019147 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe37bc-e656-4410-84e9-0270cb656cd3 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:46.020630 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8bbe37bc-e656-4410-84e9-0270cb656cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:46.035096 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe37bc-e656-4410-84e9-0270cb656cd3 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:46.039825 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c776bd-e910-4f41-8c83-639ce657ac07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:46.040452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c776bd-e910-4f41-8c83-639ce657ac07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:46.040731 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c776bd-e910-4f41-8c83-639ce657ac07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:46.041005 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c776bd-e910-4f41-8c83-639ce657ac07 admin admin] PATH_INFO: `/v3/users/3fc79694d7734de08ae43e685a26138c` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:46.042142 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c776bd-e910-4f41-8c83-639ce657ac07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fc79694d7734de08ae43e685a26138c)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:46.044415 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c776bd-e910-4f41-8c83-639ce657ac07 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:46.100637 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-91c776bd-e910-4f41-8c83-639ce657ac07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fc79694d7734de08ae43e685a26138c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:46.107628 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-91c776bd-e910-4f41-8c83-639ce657ac07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fc79694d7734de08ae43e685a26138c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:46.108293 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-91c776bd-e910-4f41-8c83-639ce657ac07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fc79694d7734de08ae43e685a26138c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:46.109000 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-91c776bd-e910-4f41-8c83-639ce657ac07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fc79694d7734de08ae43e685a26138c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:46.121875 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-91c776bd-e910-4f41-8c83-639ce657ac07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fc79694d7734de08ae43e685a26138c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:46.137529 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1444/2861] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:45:46 2022] DELETE /identity/v3/users/3fc79694d7734de08ae43e685a26138c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:46.141946 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b274054-72bb-41c1-8bdf-09b56689e351 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:46.143031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b274054-72bb-41c1-8bdf-09b56689e351 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:46.143031 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b274054-72bb-41c1-8bdf-09b56689e351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:46.163111 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9b274054-72bb-41c1-8bdf-09b56689e351 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:46.164316 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9b274054-72bb-41c1-8bdf-09b56689e351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:46.233223 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1418/2862] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:45:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:46.244011 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c776bd-e910-4f41-8c83-639ce657ac07 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:46.245476 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-91c776bd-e910-4f41-8c83-639ce657ac07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:46.306501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c776bd-e910-4f41-8c83-639ce657ac07 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:46.310474 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2a92e-eed1-4cf2-87ce-b7815a9cf7a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:46.311089 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2a92e-eed1-4cf2-87ce-b7815a9cf7a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:46.311867 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2a92e-eed1-4cf2-87ce-b7815a9cf7a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:46.312152 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2a92e-eed1-4cf2-87ce-b7815a9cf7a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:46.322771 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2a92e-eed1-4cf2-87ce-b7815a9cf7a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:46.324199 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2a92e-eed1-4cf2-87ce-b7815a9cf7a2 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:46.334553 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1445/2863] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:46.640714 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b274054-72bb-41c1-8bdf-09b56689e351 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:46.642303 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9b274054-72bb-41c1-8bdf-09b56689e351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:46.663977 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b274054-72bb-41c1-8bdf-09b56689e351 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:46.670210 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad5fd7-3868-460d-b494-984a802fecf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:46.670937 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad5fd7-3868-460d-b494-984a802fecf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:46.671389 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad5fd7-3868-460d-b494-984a802fecf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:46.671714 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad5fd7-3868-460d-b494-984a802fecf0 admin admin] PATH_INFO: `/v3/projects/03feb48be81c4865b12be5e67988a8d8` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:46.672945 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad5fd7-3868-460d-b494-984a802fecf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03feb48be81c4865b12be5e67988a8d8)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:46.675844 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad5fd7-3868-460d-b494-984a802fecf0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:46.724408 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2cad5fd7-3868-460d-b494-984a802fecf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03feb48be81c4865b12be5e67988a8d8', 'request_id': 'req-2cad5fd7-3868-460d-b494-984a802fecf0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:46.731017 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2cad5fd7-3868-460d-b494-984a802fecf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03feb48be81c4865b12be5e67988a8d8', 'request_id': 'req-2cad5fd7-3868-460d-b494-984a802fecf0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:46.739405 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2cad5fd7-3868-460d-b494-984a802fecf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03feb48be81c4865b12be5e67988a8d8', 'request_id': 'req-2cad5fd7-3868-460d-b494-984a802fecf0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:46.750087 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2cad5fd7-3868-460d-b494-984a802fecf0 admin admin] The token cache is being invalidate because project 03feb48be81c4865b12be5e67988a8d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:45:46.750401 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2cad5fd7-3868-460d-b494-984a802fecf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:46.751068 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2cad5fd7-3868-460d-b494-984a802fecf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:46.752589 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1419/2864] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:45:46 2022] DELETE /identity/v3/projects/03feb48be81c4865b12be5e67988a8d8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:46.757436 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c8eb7-e34d-4371-a63e-20f674356d37 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:46.757744 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c8eb7-e34d-4371-a63e-20f674356d37 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:46.758016 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c8eb7-e34d-4371-a63e-20f674356d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:46.777610 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b28c8eb7-e34d-4371-a63e-20f674356d37 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:46.778849 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b28c8eb7-e34d-4371-a63e-20f674356d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:46.836952 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1446/2865] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:45:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:46.846594 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c96c7-e440-4aeb-891d-966f36c03f5b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:46.846945 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c96c7-e440-4aeb-891d-966f36c03f5b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:46.847225 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c96c7-e440-4aeb-891d-966f36c03f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:46.867001 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c41c96c7-e440-4aeb-891d-966f36c03f5b None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:46.868196 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c41c96c7-e440-4aeb-891d-966f36c03f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:46.876070 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1420/2866] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:45:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:46.880087 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c8eb7-e34d-4371-a63e-20f674356d37 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:46.890650 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c8eb7-e34d-4371-a63e-20f674356d37 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:46.895750 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbfe92d-6c27-4ef7-967a-855f1f00e88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:46.896339 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbfe92d-6c27-4ef7-967a-855f1f00e88f admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:46.896621 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbfe92d-6c27-4ef7-967a-855f1f00e88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:46.896896 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbfe92d-6c27-4ef7-967a-855f1f00e88f admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:46.897541 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbfe92d-6c27-4ef7-967a-855f1f00e88f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:45:46.897853 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbfe92d-6c27-4ef7-967a-855f1f00e88f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:46.899471 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbfe92d-6c27-4ef7-967a-855f1f00e88f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:46.911633 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1447/2867] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:45:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:45:46.915690 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c96c7-e440-4aeb-891d-966f36c03f5b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:46.920534 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:46.927100 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c96c7-e440-4aeb-891d-966f36c03f5b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:46.932854 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90956ea9-359c-4d2d-91ac-608baf3e4d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:46.933457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90956ea9-359c-4d2d-91ac-608baf3e4d87 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:46.933744 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90956ea9-359c-4d2d-91ac-608baf3e4d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:46.934021 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90956ea9-359c-4d2d-91ac-608baf3e4d87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:46.934696 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90956ea9-359c-4d2d-91ac-608baf3e4d87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:46.936585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90956ea9-359c-4d2d-91ac-608baf3e4d87 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:46.963568 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1421/2868] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:45:46 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:46.967648 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbfe92d-6c27-4ef7-967a-855f1f00e88f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:46.973582 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:46.980070 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbfe92d-6c27-4ef7-967a-855f1f00e88f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:47.044640 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a8a55-a5e9-4c71-8e7c-91b23b2cdd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:47.045317 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a8a55-a5e9-4c71-8e7c-91b23b2cdd15 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:47.045518 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a8a55-a5e9-4c71-8e7c-91b23b2cdd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:47.045786 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a8a55-a5e9-4c71-8e7c-91b23b2cdd15 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:47.046472 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a8a55-a5e9-4c71-8e7c-91b23b2cdd15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:47.048384 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a8a55-a5e9-4c71-8e7c-91b23b2cdd15 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:47.070535 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1448/2869] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:45:46 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:47.074782 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90956ea9-359c-4d2d-91ac-608baf3e4d87 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:47.085084 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90956ea9-359c-4d2d-91ac-608baf3e4d87 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:47.090265 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4179d-cb53-487f-ae78-dcf7b7ce11e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:47.090864 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4179d-cb53-487f-ae78-dcf7b7ce11e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:47.091144 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4179d-cb53-487f-ae78-dcf7b7ce11e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:47.091578 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4179d-cb53-487f-ae78-dcf7b7ce11e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:47.092223 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb4179d-cb53-487f-ae78-dcf7b7ce11e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:47.094293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb4179d-cb53-487f-ae78-dcf7b7ce11e7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:47.115658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1422/2870] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:47.119749 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a8a55-a5e9-4c71-8e7c-91b23b2cdd15 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:47.130763 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a8a55-a5e9-4c71-8e7c-91b23b2cdd15 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:47.135439 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f24f77-e0b2-44b5-a6fd-9924a2f8c285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:47.136039 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f24f77-e0b2-44b5-a6fd-9924a2f8c285 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:47.136322 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f24f77-e0b2-44b5-a6fd-9924a2f8c285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:47.136626 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f24f77-e0b2-44b5-a6fd-9924a2f8c285 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:47.137271 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f24f77-e0b2-44b5-a6fd-9924a2f8c285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:47.139003 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f24f77-e0b2-44b5-a6fd-9924a2f8c285 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:47.150253 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1449/2871] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:47.154403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4179d-cb53-487f-ae78-dcf7b7ce11e7 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:47.164694 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4179d-cb53-487f-ae78-dcf7b7ce11e7 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:47.169587 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b82d4a-54e4-412c-af31-67c93cc976e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:47.170235 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b82d4a-54e4-412c-af31-67c93cc976e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:47.170548 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b82d4a-54e4-412c-af31-67c93cc976e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:47.170827 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b82d4a-54e4-412c-af31-67c93cc976e1 admin admin] PATH_INFO: `/v3/projects/1f62976899a74162b21efe5580758fd9/users/fac40ee8cdc9495aa1c94f483f6fd6c5/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:47.185392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b82d4a-54e4-412c-af31-67c93cc976e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f62976899a74162b21efe5580758fd9, user_id=fac40ee8cdc9495aa1c94f483f6fd6c5, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:47.188073 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b82d4a-54e4-412c-af31-67c93cc976e1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:47.202438 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1423/2872] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:47 2022] PUT /identity/v3/projects/1f62976899a74162b21efe5580758fd9/users/fac40ee8cdc9495aa1c94f483f6fd6c5/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:47.208117 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f38ea-54c6-4f28-91a2-6e561e01e504 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:47.208421 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f38ea-54c6-4f28-91a2-6e561e01e504 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:47.208690 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f38ea-54c6-4f28-91a2-6e561e01e504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:47.227433 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7f9f38ea-54c6-4f28-91a2-6e561e01e504 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:47.228615 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7f9f38ea-54c6-4f28-91a2-6e561e01e504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:47.286705 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1450/2873] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:45:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:47.296559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b82d4a-54e4-412c-af31-67c93cc976e1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:47.298260 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d1b82d4a-54e4-412c-af31-67c93cc976e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:47.354504 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b82d4a-54e4-412c-af31-67c93cc976e1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:47.358562 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ead6d-730e-438d-97cd-d5f6819abf76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:47.359160 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ead6d-730e-438d-97cd-d5f6819abf76 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:47.359443 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ead6d-730e-438d-97cd-d5f6819abf76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:47.359719 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ead6d-730e-438d-97cd-d5f6819abf76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:47.368751 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ead6d-730e-438d-97cd-d5f6819abf76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:47.370172 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ead6d-730e-438d-97cd-d5f6819abf76 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:47.380233 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1424/2874] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:47.763560 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f38ea-54c6-4f28-91a2-6e561e01e504 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:47.765489 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7f9f38ea-54c6-4f28-91a2-6e561e01e504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:47.779081 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f38ea-54c6-4f28-91a2-6e561e01e504 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:47.782896 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a09fb-6d83-4f79-bd18-5ca88d274937 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:47.783532 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a09fb-6d83-4f79-bd18-5ca88d274937 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:47.783816 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a09fb-6d83-4f79-bd18-5ca88d274937 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:47.784089 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a09fb-6d83-4f79-bd18-5ca88d274937 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:47.785839 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c47a09fb-6d83-4f79-bd18-5ca88d274937 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:47.863309 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47a09fb-6d83-4f79-bd18-5ca88d274937 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:47.865179 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47a09fb-6d83-4f79-bd18-5ca88d274937 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:47.877183 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1451/2875] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:51.546761 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c361501-aa4c-4085-a8d6-6c54b7af7ebe None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:51.547424 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c361501-aa4c-4085-a8d6-6c54b7af7ebe None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:51.547424 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c361501-aa4c-4085-a8d6-6c54b7af7ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:51.583722 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0c361501-aa4c-4085-a8d6-6c54b7af7ebe None None] MFA Rules not processed for user `fac40ee8cdc9495aa1c94f483f6fd6c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:51.584969 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0c361501-aa4c-4085-a8d6-6c54b7af7ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:51.661948 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1425/2876] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:45:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:51.666532 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a09fb-6d83-4f79-bd18-5ca88d274937 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:51.678973 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a09fb-6d83-4f79-bd18-5ca88d274937 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:51.683554 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7941662-5deb-4f13-b073-3b02b7668c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:51.684205 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7941662-5deb-4f13-b073-3b02b7668c8d admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:51.684487 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7941662-5deb-4f13-b073-3b02b7668c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:51.684761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7941662-5deb-4f13-b073-3b02b7668c8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:51.685413 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7941662-5deb-4f13-b073-3b02b7668c8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:51.687407 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7941662-5deb-4f13-b073-3b02b7668c8d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:51.692358 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:51.723580 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1452/2877] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:45:51 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:51.727697 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c361501-aa4c-4085-a8d6-6c54b7af7ebe None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:51.739506 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c361501-aa4c-4085-a8d6-6c54b7af7ebe None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:51.775757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:51.803638 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0575cc-f9e0-4413-9550-266503b3b936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:51.804373 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0575cc-f9e0-4413-9550-266503b3b936 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:51.804722 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0575cc-f9e0-4413-9550-266503b3b936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:51.805072 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0575cc-f9e0-4413-9550-266503b3b936 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:51.805789 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0575cc-f9e0-4413-9550-266503b3b936 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:51.807867 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0575cc-f9e0-4413-9550-266503b3b936 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:51.831122 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1426/2878] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:45:51 2022] POST /identity/v3/users => generated 298 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:45:51.835458 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7941662-5deb-4f13-b073-3b02b7668c8d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:51.846780 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7941662-5deb-4f13-b073-3b02b7668c8d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:51.855601 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86576d0-a95c-4299-957f-5e0c5498e8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:51.856226 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86576d0-a95c-4299-957f-5e0c5498e8e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:51.856514 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86576d0-a95c-4299-957f-5e0c5498e8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:51.856790 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86576d0-a95c-4299-957f-5e0c5498e8e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:51.857585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86576d0-a95c-4299-957f-5e0c5498e8e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:51.859241 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86576d0-a95c-4299-957f-5e0c5498e8e0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:51.874013 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1453/2879] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:45:51.879041 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0575cc-f9e0-4413-9550-266503b3b936 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:51.889310 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0575cc-f9e0-4413-9550-266503b3b936 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:51.894406 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ecea7-a502-4673-8bf1-f199ff2ff708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:51.895082 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ecea7-a502-4673-8bf1-f199ff2ff708 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:51.895386 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ecea7-a502-4673-8bf1-f199ff2ff708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:51.895794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ecea7-a502-4673-8bf1-f199ff2ff708 admin admin] PATH_INFO: `/v3/projects/81558e2e92b24d02b1040a545d867a36/users/a69e65a5607f4d14b3980e9ebaab2e08/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:51.915191 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ecea7-a502-4673-8bf1-f199ff2ff708 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81558e2e92b24d02b1040a545d867a36, user_id=a69e65a5607f4d14b3980e9ebaab2e08, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:51.918678 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ecea7-a502-4673-8bf1-f199ff2ff708 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:51.935555 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1427/2880] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:51 2022] PUT /identity/v3/projects/81558e2e92b24d02b1040a545d867a36/users/a69e65a5607f4d14b3980e9ebaab2e08/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:51.939595 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86576d0-a95c-4299-957f-5e0c5498e8e0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:51.951018 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86576d0-a95c-4299-957f-5e0c5498e8e0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:52.015401 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb2f0d-406d-4dab-a546-9709a6d8dbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:52.016029 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb2f0d-406d-4dab-a546-9709a6d8dbc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:52.016320 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb2f0d-406d-4dab-a546-9709a6d8dbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:52.016623 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb2f0d-406d-4dab-a546-9709a6d8dbc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:52.017288 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffb2f0d-406d-4dab-a546-9709a6d8dbc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:52.019313 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffb2f0d-406d-4dab-a546-9709a6d8dbc8 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:52.034282 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1454/2881] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:52.038496 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ecea7-a502-4673-8bf1-f199ff2ff708 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:52.052193 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ecea7-a502-4673-8bf1-f199ff2ff708 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:52.058678 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc9490-99d5-427a-ba28-61eb5ab64209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:52.059426 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc9490-99d5-427a-ba28-61eb5ab64209 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:52.059714 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc9490-99d5-427a-ba28-61eb5ab64209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:52.060084 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc9490-99d5-427a-ba28-61eb5ab64209 admin admin] PATH_INFO: `/v3/projects/81558e2e92b24d02b1040a545d867a36/users/a69e65a5607f4d14b3980e9ebaab2e08/roles/ff293acb8551463aafe6970230042b89` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:52.062528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fc9490-99d5-427a-ba28-61eb5ab64209 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81558e2e92b24d02b1040a545d867a36, user_id=a69e65a5607f4d14b3980e9ebaab2e08, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:52.065988 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fc9490-99d5-427a-ba28-61eb5ab64209 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:52.077263 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1428/2882] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:52 2022] PUT /identity/v3/projects/81558e2e92b24d02b1040a545d867a36/users/a69e65a5607f4d14b3980e9ebaab2e08/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:56.809067 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12d9bd-98c3-4782-a76b-2bd6361d001c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:56.809612 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12d9bd-98c3-4782-a76b-2bd6361d001c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:56.809612 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12d9bd-98c3-4782-a76b-2bd6361d001c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:56.842022 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-1c12d9bd-98c3-4782-a76b-2bd6361d001c None None] MFA Rules not processed for user `a69e65a5607f4d14b3980e9ebaab2e08`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:56.843344 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1c12d9bd-98c3-4782-a76b-2bd6361d001c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:56.933581 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1455/2883] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:45:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:56.946849 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc9490-99d5-427a-ba28-61eb5ab64209 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:56.948901 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-97fc9490-99d5-427a-ba28-61eb5ab64209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:57.066522 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc9490-99d5-427a-ba28-61eb5ab64209 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:57.070637 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c8435-c135-4129-9923-e6b2877716d5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a2c872129414105a112661cc2b477e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:57.071476 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c8435-c135-4129-9923-e6b2877716d5 service glance] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:57.072019 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c8435-c135-4129-9923-e6b2877716d5 service glance] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:57.072462 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c8435-c135-4129-9923-e6b2877716d5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:57.147988 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c8435-c135-4129-9923-e6b2877716d5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:57.150105 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c8435-c135-4129-9923-e6b2877716d5 service glance] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:57.165969 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1429/2884] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:57.304139 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0c820-0663-4217-8574-d9a83f7916a3 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:57.304500 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0c820-0663-4217-8574-d9a83f7916a3 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:57.304896 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0c820-0663-4217-8574-d9a83f7916a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:57.330502 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-cfe0c820-0663-4217-8574-d9a83f7916a3 None None] MFA Rules not processed for user `dd7b98536fa445cebd552fcbf54b757d`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:57.331805 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-cfe0c820-0663-4217-8574-d9a83f7916a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:57.390367 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:57.403894 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1456/2885] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Sep 1 16:45:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:45:57.460978 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c8435-c135-4129-9923-e6b2877716d5 service glance] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:57.462566 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ec7c8435-c135-4129-9923-e6b2877716d5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:57.492736 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:57.535679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c8435-c135-4129-9923-e6b2877716d5 service glance] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:57.540694 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27661134-b507-458e-aa50-33794eb8f8fc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e789695517c1437fb224f5a6195655f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:57.541357 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27661134-b507-458e-aa50-33794eb8f8fc service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:57.541644 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27661134-b507-458e-aa50-33794eb8f8fc service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:57.541914 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27661134-b507-458e-aa50-33794eb8f8fc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:57.551256 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27661134-b507-458e-aa50-33794eb8f8fc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:57.552860 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27661134-b507-458e-aa50-33794eb8f8fc service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:57.560834 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1430/2886] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Sep 1 16:45:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:46:06.235353 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c1184-f7c9-4fa1-8351-58ff16ccd295 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:06.235847 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c1184-f7c9-4fa1-8351-58ff16ccd295 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:06.235950 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c1184-f7c9-4fa1-8351-58ff16ccd295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:06.254798 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-dd4c1184-f7c9-4fa1-8351-58ff16ccd295 None None] MFA Rules not processed for user `fac40ee8cdc9495aa1c94f483f6fd6c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:06.255997 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-dd4c1184-f7c9-4fa1-8351-58ff16ccd295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:06.262811 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1457/2887] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:46:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:07.557582 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=21 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:08.328177 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21275475-fa5c-412c-b1e8-cac550c8dad1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:08.328472 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21275475-fa5c-412c-b1e8-cac550c8dad1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:08.328752 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21275475-fa5c-412c-b1e8-cac550c8dad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:08.347609 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-21275475-fa5c-412c-b1e8-cac550c8dad1 None None] MFA Rules not processed for user `dd7b98536fa445cebd552fcbf54b757d`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:08.348824 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-21275475-fa5c-412c-b1e8-cac550c8dad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:08.356051 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1431/2888] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Sep 1 16:46:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:08.369249 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c1184-f7c9-4fa1-8351-58ff16ccd295 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:08.381880 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c1184-f7c9-4fa1-8351-58ff16ccd295 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:08.387040 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5fb003-45d7-4e1d-902f-176045047974 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e789695517c1437fb224f5a6195655f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:08.387674 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fb003-45d7-4e1d-902f-176045047974 service swift] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:08.387956 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fb003-45d7-4e1d-902f-176045047974 service swift] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:08.388230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fb003-45d7-4e1d-902f-176045047974 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:08.396577 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5fb003-45d7-4e1d-902f-176045047974 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:08.398265 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5fb003-45d7-4e1d-902f-176045047974 service swift] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:08.406479 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1458/2889] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Sep 1 16:46:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:46:18.342078 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:18.403149 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=9 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:23.154889 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21275475-fa5c-412c-b1e8-cac550c8dad1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:23.222167 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21275475-fa5c-412c-b1e8-cac550c8dad1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:23.227217 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e133fe-cce0-4a83-b1b5-7f7f65142cf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:23.227835 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e133fe-cce0-4a83-b1b5-7f7f65142cf9 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:23.228224 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e133fe-cce0-4a83-b1b5-7f7f65142cf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:23.228439 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e133fe-cce0-4a83-b1b5-7f7f65142cf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:23.231048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c7e133fe-cce0-4a83-b1b5-7f7f65142cf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:23.242069 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e133fe-cce0-4a83-b1b5-7f7f65142cf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:23.243674 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e133fe-cce0-4a83-b1b5-7f7f65142cf9 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:23.253457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1432/2890] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:33.249288 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=26 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:53.700924 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5fb003-45d7-4e1d-902f-176045047974 service swift] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:53.710481 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5fb003-45d7-4e1d-902f-176045047974 service swift] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:53.715392 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce389257-6ef8-4945-8a58-05d0ac89a902 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:53.716016 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce389257-6ef8-4945-8a58-05d0ac89a902 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:53.716292 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce389257-6ef8-4945-8a58-05d0ac89a902 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:53.716562 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce389257-6ef8-4945-8a58-05d0ac89a902 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:53.727353 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce389257-6ef8-4945-8a58-05d0ac89a902 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:53.729327 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce389257-6ef8-4945-8a58-05d0ac89a902 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:53.739575 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1459/2891] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:53 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:57.004393 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e133fe-cce0-4a83-b1b5-7f7f65142cf9 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:57.005866 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c7e133fe-cce0-4a83-b1b5-7f7f65142cf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:57.118273 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e133fe-cce0-4a83-b1b5-7f7f65142cf9 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:57.124044 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a7962-3c77-4682-a9d6-5e1109374fce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:57.124663 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a7962-3c77-4682-a9d6-5e1109374fce service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:57.125016 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a7962-3c77-4682-a9d6-5e1109374fce service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:57.125220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a7962-3c77-4682-a9d6-5e1109374fce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:57.127114 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-026a7962-3c77-4682-a9d6-5e1109374fce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:57.141524 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a7962-3c77-4682-a9d6-5e1109374fce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:57.143326 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a7962-3c77-4682-a9d6-5e1109374fce service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:57.155535 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1433/2892] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:47:03.735591 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=6 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:47:07.151096 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=34 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:47:11.941787 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7a149-d7e2-416d-b3a3-ffe35601463b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:11.942319 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7a149-d7e2-416d-b3a3-ffe35601463b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:11.942461 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7a149-d7e2-416d-b3a3-ffe35601463b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:11.963063 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-c2e7a149-d7e2-416d-b3a3-ffe35601463b None None] MFA Rules not processed for user `e8f521436e924bf6a4571d3aeda481cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:47:11.964288 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c2e7a149-d7e2-416d-b3a3-ffe35601463b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:12.019841 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1460/2893] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:47:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:47:22.013187 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=25 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:47:46.261670 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a7962-3c77-4682-a9d6-5e1109374fce service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:46.271301 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a7962-3c77-4682-a9d6-5e1109374fce service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:46.275019 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26172f60-e468-43b3-95d8-0c280005464b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:46.275619 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26172f60-e468-43b3-95d8-0c280005464b service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:46.275903 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26172f60-e468-43b3-95d8-0c280005464b service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:46.276180 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26172f60-e468-43b3-95d8-0c280005464b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:46.277937 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-26172f60-e468-43b3-95d8-0c280005464b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:46.338331 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26172f60-e468-43b3-95d8-0c280005464b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:46.339716 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26172f60-e468-43b3-95d8-0c280005464b service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:46.349420 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1434/2894] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:47:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:47:54.058501 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7a149-d7e2-416d-b3a3-ffe35601463b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:54.069694 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7a149-d7e2-416d-b3a3-ffe35601463b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:54.074088 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10526ae-a09a-4359-9b1b-40b5d4af4405 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:54.074740 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10526ae-a09a-4359-9b1b-40b5d4af4405 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:54.075019 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10526ae-a09a-4359-9b1b-40b5d4af4405 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:54.075290 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10526ae-a09a-4359-9b1b-40b5d4af4405 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:54.077669 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c10526ae-a09a-4359-9b1b-40b5d4af4405 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:54.150060 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10526ae-a09a-4359-9b1b-40b5d4af4405 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:54.151643 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10526ae-a09a-4359-9b1b-40b5d4af4405 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:54.166041 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1461/2895] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:47:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:47:56.344923 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=28 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:48:01.113679 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26172f60-e468-43b3-95d8-0c280005464b service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:01.116048 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-26172f60-e468-43b3-95d8-0c280005464b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:01.193921 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10526ae-a09a-4359-9b1b-40b5d4af4405 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:01.196399 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c10526ae-a09a-4359-9b1b-40b5d4af4405 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:01.217491 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10526ae-a09a-4359-9b1b-40b5d4af4405 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:01.217680 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26172f60-e468-43b3-95d8-0c280005464b service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:01.227838 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6912b-f8bf-4b76-936c-1b371f7d0e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:01.228688 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6912b-f8bf-4b76-936c-1b371f7d0e6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:01.229102 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6912b-f8bf-4b76-936c-1b371f7d0e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:01.229739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de8d5a-b4ac-4181-9125-90f8f38c11a5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12302cdb2854b0684179647a6dadf6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:01.231203 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6912b-f8bf-4b76-936c-1b371f7d0e6e admin admin] PATH_INFO: `/v3/users/d0b60450d50b4fbaa9a85dd39643e403` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:01.231313 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de8d5a-b4ac-4181-9125-90f8f38c11a5 service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:01.231313 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de8d5a-b4ac-4181-9125-90f8f38c11a5 service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:01.231313 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de8d5a-b4ac-4181-9125-90f8f38c11a5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:01.231610 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6912b-f8bf-4b76-936c-1b371f7d0e6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0b60450d50b4fbaa9a85dd39643e403)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:01.233103 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-27de8d5a-b4ac-4181-9125-90f8f38c11a5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:01.234091 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6912b-f8bf-4b76-936c-1b371f7d0e6e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:01.247144 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27de8d5a-b4ac-4181-9125-90f8f38c11a5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:01.248969 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27de8d5a-b4ac-4181-9125-90f8f38c11a5 service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:01.261111 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1435/2896] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:48:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:48:01.303155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-27b6912b-f8bf-4b76-936c-1b371f7d0e6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0b60450d50b4fbaa9a85dd39643e403'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:01.311764 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-27b6912b-f8bf-4b76-936c-1b371f7d0e6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0b60450d50b4fbaa9a85dd39643e403'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:01.312510 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-27b6912b-f8bf-4b76-936c-1b371f7d0e6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0b60450d50b4fbaa9a85dd39643e403'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:01.313124 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-27b6912b-f8bf-4b76-936c-1b371f7d0e6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0b60450d50b4fbaa9a85dd39643e403'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:01.333189 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-27b6912b-f8bf-4b76-936c-1b371f7d0e6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0b60450d50b4fbaa9a85dd39643e403'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:01.346535 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1462/2897] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:48:01 2022] DELETE /identity/v3/users/d0b60450d50b4fbaa9a85dd39643e403 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:48:01.545026 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de8d5a-b4ac-4181-9125-90f8f38c11a5 service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:01.560828 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de8d5a-b4ac-4181-9125-90f8f38c11a5 service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:01.631212 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711fb9c-fc1c-4211-8076-d6e4ecc2048e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:01.631853 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711fb9c-fc1c-4211-8076-d6e4ecc2048e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:01.632139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711fb9c-fc1c-4211-8076-d6e4ecc2048e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:01.632419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711fb9c-fc1c-4211-8076-d6e4ecc2048e admin admin] PATH_INFO: `/v3/projects/bde90cf98c1140f298e9cc8db9c8932f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:01.633585 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2711fb9c-fc1c-4211-8076-d6e4ecc2048e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bde90cf98c1140f298e9cc8db9c8932f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:01.635605 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2711fb9c-fc1c-4211-8076-d6e4ecc2048e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:01.694870 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2711fb9c-fc1c-4211-8076-d6e4ecc2048e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bde90cf98c1140f298e9cc8db9c8932f', 'request_id': 'req-2711fb9c-fc1c-4211-8076-d6e4ecc2048e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:01.704392 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2711fb9c-fc1c-4211-8076-d6e4ecc2048e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bde90cf98c1140f298e9cc8db9c8932f', 'request_id': 'req-2711fb9c-fc1c-4211-8076-d6e4ecc2048e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:01.715584 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2711fb9c-fc1c-4211-8076-d6e4ecc2048e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bde90cf98c1140f298e9cc8db9c8932f', 'request_id': 'req-2711fb9c-fc1c-4211-8076-d6e4ecc2048e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:01.735311 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2711fb9c-fc1c-4211-8076-d6e4ecc2048e admin admin] The token cache is being invalidate because project bde90cf98c1140f298e9cc8db9c8932f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:48:01.735514 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2711fb9c-fc1c-4211-8076-d6e4ecc2048e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:01.736175 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2711fb9c-fc1c-4211-8076-d6e4ecc2048e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:01.737661 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1436/2898] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:48:01 2022] DELETE /identity/v3/projects/bde90cf98c1140f298e9cc8db9c8932f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:48:01.742726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6912b-f8bf-4b76-936c-1b371f7d0e6e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:01.744744 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-27b6912b-f8bf-4b76-936c-1b371f7d0e6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:01.767158 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6912b-f8bf-4b76-936c-1b371f7d0e6e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:01.835931 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5357fad-1502-46f9-b97c-b66744ee85d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:01.837374 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5357fad-1502-46f9-b97c-b66744ee85d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:01.837655 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5357fad-1502-46f9-b97c-b66744ee85d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:01.837923 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5357fad-1502-46f9-b97c-b66744ee85d8 admin admin] PATH_INFO: `/v3/users/5b4ce389b7714919abd2111411c36651` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:01.839129 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5357fad-1502-46f9-b97c-b66744ee85d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b4ce389b7714919abd2111411c36651)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:01.841040 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5357fad-1502-46f9-b97c-b66744ee85d8 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:01.849159 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:48:01.907453 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b5357fad-1502-46f9-b97c-b66744ee85d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b4ce389b7714919abd2111411c36651'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:01.923285 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b5357fad-1502-46f9-b97c-b66744ee85d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b4ce389b7714919abd2111411c36651'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:01.924151 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b5357fad-1502-46f9-b97c-b66744ee85d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b4ce389b7714919abd2111411c36651'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:01.924821 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b5357fad-1502-46f9-b97c-b66744ee85d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b4ce389b7714919abd2111411c36651'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:01.947587 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b5357fad-1502-46f9-b97c-b66744ee85d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b4ce389b7714919abd2111411c36651'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:01.971575 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1463/2899] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:48:01 2022] DELETE /identity/v3/users/5b4ce389b7714919abd2111411c36651 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:48:02.294454 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711fb9c-fc1c-4211-8076-d6e4ecc2048e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:02.296310 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2711fb9c-fc1c-4211-8076-d6e4ecc2048e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:02.318165 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711fb9c-fc1c-4211-8076-d6e4ecc2048e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:02.342129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:48:02.406124 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bdcd6e-4bf8-4458-bf8c-c2932cbe7606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:02.406847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdcd6e-4bf8-4458-bf8c-c2932cbe7606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:02.407130 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdcd6e-4bf8-4458-bf8c-c2932cbe7606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:02.407403 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdcd6e-4bf8-4458-bf8c-c2932cbe7606 admin admin] PATH_INFO: `/v3/projects/61201dd344504df4958112b9b6eff04c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:02.408713 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bdcd6e-4bf8-4458-bf8c-c2932cbe7606 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61201dd344504df4958112b9b6eff04c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:02.410837 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bdcd6e-4bf8-4458-bf8c-c2932cbe7606 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:02.467678 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f6bdcd6e-4bf8-4458-bf8c-c2932cbe7606 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61201dd344504df4958112b9b6eff04c', 'request_id': 'req-f6bdcd6e-4bf8-4458-bf8c-c2932cbe7606'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:02.474382 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f6bdcd6e-4bf8-4458-bf8c-c2932cbe7606 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61201dd344504df4958112b9b6eff04c', 'request_id': 'req-f6bdcd6e-4bf8-4458-bf8c-c2932cbe7606'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:02.483296 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f6bdcd6e-4bf8-4458-bf8c-c2932cbe7606 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61201dd344504df4958112b9b6eff04c', 'request_id': 'req-f6bdcd6e-4bf8-4458-bf8c-c2932cbe7606'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:02.495528 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f6bdcd6e-4bf8-4458-bf8c-c2932cbe7606 admin admin] The token cache is being invalidate because project 61201dd344504df4958112b9b6eff04c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:48:02.495881 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f6bdcd6e-4bf8-4458-bf8c-c2932cbe7606 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:02.496636 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f6bdcd6e-4bf8-4458-bf8c-c2932cbe7606 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:02.498593 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1437/2900] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:48:02 2022] DELETE /identity/v3/projects/61201dd344504df4958112b9b6eff04c => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:48:02.504044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7af471-2368-4e3a-a9f9-dd959772c9d1 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:02.504920 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7af471-2368-4e3a-a9f9-dd959772c9d1 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:02.505212 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7af471-2368-4e3a-a9f9-dd959772c9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:02.526758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-1f7af471-2368-4e3a-a9f9-dd959772c9d1 None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:48:02.528237 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1f7af471-2368-4e3a-a9f9-dd959772c9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:02.603677 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1464/2901] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:48:02.611787 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab91339-3940-46a9-bb12-072ccd59280a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:02.612067 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab91339-3940-46a9-bb12-072ccd59280a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:02.612342 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab91339-3940-46a9-bb12-072ccd59280a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:02.632682 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-aab91339-3940-46a9-bb12-072ccd59280a None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:48:02.633931 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-aab91339-3940-46a9-bb12-072ccd59280a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:02.642038 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1438/2902] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:48:02.646859 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7af471-2368-4e3a-a9f9-dd959772c9d1 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:02.657877 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7af471-2368-4e3a-a9f9-dd959772c9d1 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:02.662664 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bdc562-9ea9-44e5-ac48-98a1b4ec797e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:02.663277 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdc562-9ea9-44e5-ac48-98a1b4ec797e admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:02.663555 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdc562-9ea9-44e5-ac48-98a1b4ec797e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:02.663847 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdc562-9ea9-44e5-ac48-98a1b4ec797e admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:02.665105 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bdc562-9ea9-44e5-ac48-98a1b4ec797e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:48:02.665408 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bdc562-9ea9-44e5-ac48-98a1b4ec797e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:02.667047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bdc562-9ea9-44e5-ac48-98a1b4ec797e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:02.679694 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1465/2903] 10.222.0.58 () {58 vars in 1153 bytes} [Thu Sep 1 16:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:48:02.684684 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab91339-3940-46a9-bb12-072ccd59280a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:02.694225 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab91339-3940-46a9-bb12-072ccd59280a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:02.699003 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557faa8-59fe-46d5-86e1-3a3d2b658ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:02.699609 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557faa8-59fe-46d5-86e1-3a3d2b658ddf admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:02.699888 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557faa8-59fe-46d5-86e1-3a3d2b658ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:02.700158 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557faa8-59fe-46d5-86e1-3a3d2b658ddf admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:02.700791 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0557faa8-59fe-46d5-86e1-3a3d2b658ddf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:02.702729 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0557faa8-59fe-46d5-86e1-3a3d2b658ddf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:02.735077 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1439/2904] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:48:02 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:48:02.739296 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bdc562-9ea9-44e5-ac48-98a1b4ec797e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:02.750044 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bdc562-9ea9-44e5-ac48-98a1b4ec797e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:02.810987 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3991b61-7b38-4792-9d30-ab2e362c4f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:02.811888 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3991b61-7b38-4792-9d30-ab2e362c4f99 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:02.812202 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3991b61-7b38-4792-9d30-ab2e362c4f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:02.812689 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3991b61-7b38-4792-9d30-ab2e362c4f99 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:02.813628 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3991b61-7b38-4792-9d30-ab2e362c4f99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:02.815751 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3991b61-7b38-4792-9d30-ab2e362c4f99 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:02.839622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1466/2905] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:48:02 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:48:02.845175 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557faa8-59fe-46d5-86e1-3a3d2b658ddf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:02.855677 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557faa8-59fe-46d5-86e1-3a3d2b658ddf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:02.860315 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495de15c-f50b-4ed9-9677-7fc7a39824db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:02.860916 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495de15c-f50b-4ed9-9677-7fc7a39824db admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:02.861195 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495de15c-f50b-4ed9-9677-7fc7a39824db admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:02.861466 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495de15c-f50b-4ed9-9677-7fc7a39824db admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:02.862111 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495de15c-f50b-4ed9-9677-7fc7a39824db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:02.863748 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495de15c-f50b-4ed9-9677-7fc7a39824db admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:02.874355 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1440/2906] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:48:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:48:02.879150 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3991b61-7b38-4792-9d30-ab2e362c4f99 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:02.885007 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:48:02.898599 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3991b61-7b38-4792-9d30-ab2e362c4f99 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:02.904998 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4d6c2-b9ba-46c5-a11d-ea822b56f61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:02.905603 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4d6c2-b9ba-46c5-a11d-ea822b56f61e admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:02.905877 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4d6c2-b9ba-46c5-a11d-ea822b56f61e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:02.906168 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4d6c2-b9ba-46c5-a11d-ea822b56f61e admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:02.906830 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d4d6c2-b9ba-46c5-a11d-ea822b56f61e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:02.908438 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d4d6c2-b9ba-46c5-a11d-ea822b56f61e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:02.919999 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1467/2907] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:48:02 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:48:02.924536 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495de15c-f50b-4ed9-9677-7fc7a39824db admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:02.934920 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495de15c-f50b-4ed9-9677-7fc7a39824db admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:02.939478 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bceb44d-06e6-4a20-9ff3-26fca2b15368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:02.940131 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bceb44d-06e6-4a20-9ff3-26fca2b15368 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:02.940410 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bceb44d-06e6-4a20-9ff3-26fca2b15368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:02.940687 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bceb44d-06e6-4a20-9ff3-26fca2b15368 admin admin] PATH_INFO: `/v3/projects/20a271a8da6d423db38d5730006af839/users/8fa0f309a0b749b1a782d489f1f352af/roles/42db5f955f2644b1bb22b0a68f8cd829` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:02.957591 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bceb44d-06e6-4a20-9ff3-26fca2b15368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20a271a8da6d423db38d5730006af839, user_id=8fa0f309a0b749b1a782d489f1f352af, role_id=42db5f955f2644b1bb22b0a68f8cd829)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:02.960293 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bceb44d-06e6-4a20-9ff3-26fca2b15368 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:02.975238 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1441/2908] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:48:02 2022] PUT /identity/v3/projects/20a271a8da6d423db38d5730006af839/users/8fa0f309a0b749b1a782d489f1f352af/roles/42db5f955f2644b1bb22b0a68f8cd829 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:48:02.980452 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b39cb-45f4-461b-b2dc-08e05088410d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:02.980741 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b39cb-45f4-461b-b2dc-08e05088410d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:02.981085 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b39cb-45f4-461b-b2dc-08e05088410d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:03.003334 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-dd7b39cb-45f4-461b-b2dc-08e05088410d None None] MFA Rules not processed for user `9fe3a5506b35415a9373071c0b976e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:48:03.004551 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-dd7b39cb-45f4-461b-b2dc-08e05088410d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:03.078025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1468/2909] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:48:03.089809 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bceb44d-06e6-4a20-9ff3-26fca2b15368 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:03.092180 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9bceb44d-06e6-4a20-9ff3-26fca2b15368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:03.155099 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:48:03.161665 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bceb44d-06e6-4a20-9ff3-26fca2b15368 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:03.169598 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d1bf7-85bf-4872-baea-8d45bce414ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:03.170489 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d1bf7-85bf-4872-baea-8d45bce414ee service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:03.171132 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d1bf7-85bf-4872-baea-8d45bce414ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:03.171538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d1bf7-85bf-4872-baea-8d45bce414ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:03.182693 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d1bf7-85bf-4872-baea-8d45bce414ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:03.184154 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d1bf7-85bf-4872-baea-8d45bce414ee service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:03.196367 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1442/2910] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:48:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:48:07.667491 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633f367-3b35-4b06-9229-338634cca440 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:07.668384 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633f367-3b35-4b06-9229-338634cca440 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:07.668831 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633f367-3b35-4b06-9229-338634cca440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:07.702605 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-0633f367-3b35-4b06-9229-338634cca440 None None] MFA Rules not processed for user `8fa0f309a0b749b1a782d489f1f352af`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:48:07.703766 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0633f367-3b35-4b06-9229-338634cca440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:07.784270 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1469/2911] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:48:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:48:07.789868 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d1bf7-85bf-4872-baea-8d45bce414ee service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:07.800495 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d1bf7-85bf-4872-baea-8d45bce414ee service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:07.805891 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f7f3c-c527-4690-8697-7564c06bad07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:07.806704 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f7f3c-c527-4690-8697-7564c06bad07 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:07.811666 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f7f3c-c527-4690-8697-7564c06bad07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:07.811666 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f7f3c-c527-4690-8697-7564c06bad07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:07.811666 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f7f3c-c527-4690-8697-7564c06bad07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:07.811666 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f7f3c-c527-4690-8697-7564c06bad07 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:07.845550 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1443/2912] 10.222.0.58 () {60 vars in 1153 bytes} [Thu Sep 1 16:48:07 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:48:07.849627 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633f367-3b35-4b06-9229-338634cca440 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:07.861561 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633f367-3b35-4b06-9229-338634cca440 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:07.950600 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec7005-0f33-497b-9dd1-e40dcf43b938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:07.951217 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec7005-0f33-497b-9dd1-e40dcf43b938 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:07.951495 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec7005-0f33-497b-9dd1-e40dcf43b938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:07.951765 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec7005-0f33-497b-9dd1-e40dcf43b938 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:07.952415 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec7005-0f33-497b-9dd1-e40dcf43b938 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:07.954457 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec7005-0f33-497b-9dd1-e40dcf43b938 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:07.984183 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1470/2913] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Sep 1 16:48:07 2022] POST /identity/v3/users => generated 295 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:48:07.988308 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f7f3c-c527-4690-8697-7564c06bad07 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:07.999842 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f7f3c-c527-4690-8697-7564c06bad07 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:08.005007 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f726f-bf3b-4bbc-9454-13cb0cb600e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:08.005523 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f726f-bf3b-4bbc-9454-13cb0cb600e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:08.005805 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f726f-bf3b-4bbc-9454-13cb0cb600e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:08.006073 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f726f-bf3b-4bbc-9454-13cb0cb600e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:08.006766 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f726f-bf3b-4bbc-9454-13cb0cb600e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:08.009136 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f726f-bf3b-4bbc-9454-13cb0cb600e0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:08.023101 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1444/2914] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:48:08.027371 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec7005-0f33-497b-9dd1-e40dcf43b938 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:08.043118 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec7005-0f33-497b-9dd1-e40dcf43b938 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:08.048828 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e14ba8-28fa-417f-9da1-e2cb36e27957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:08.049968 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e14ba8-28fa-417f-9da1-e2cb36e27957 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:08.050345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e14ba8-28fa-417f-9da1-e2cb36e27957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:08.050726 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e14ba8-28fa-417f-9da1-e2cb36e27957 admin admin] PATH_INFO: `/v3/projects/defb3d3830774ddaa4e7e01012a817f6/users/0f3194718cf44dbeb218e29c0e69a3df/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:08.064294 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:48:08.077965 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e14ba8-28fa-417f-9da1-e2cb36e27957 admin admin] RBAC: Authorizing `identity:create_grant(project_id=defb3d3830774ddaa4e7e01012a817f6, user_id=0f3194718cf44dbeb218e29c0e69a3df, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:08.080683 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e14ba8-28fa-417f-9da1-e2cb36e27957 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:08.096156 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1471/2915] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:48:08 2022] PUT /identity/v3/projects/defb3d3830774ddaa4e7e01012a817f6/users/0f3194718cf44dbeb218e29c0e69a3df/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:48:08.100188 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f726f-bf3b-4bbc-9454-13cb0cb600e0 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:08.111129 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f726f-bf3b-4bbc-9454-13cb0cb600e0 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:08.175292 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f1267-9ba8-4b66-ba11-268a0b7287dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:08.175908 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f1267-9ba8-4b66-ba11-268a0b7287dd admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:08.176189 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f1267-9ba8-4b66-ba11-268a0b7287dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:08.176462 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f1267-9ba8-4b66-ba11-268a0b7287dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:08.177110 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f1267-9ba8-4b66-ba11-268a0b7287dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:08.178762 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f1267-9ba8-4b66-ba11-268a0b7287dd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:08.191591 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1445/2916] 10.222.0.58 () {58 vars in 1122 bytes} [Thu Sep 1 16:48:08 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:48:08.196029 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e14ba8-28fa-417f-9da1-e2cb36e27957 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:08.207188 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e14ba8-28fa-417f-9da1-e2cb36e27957 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:08.211824 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3227e-e182-4e3d-91e8-c74909761220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:08.212475 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3227e-e182-4e3d-91e8-c74909761220 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:08.212753 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3227e-e182-4e3d-91e8-c74909761220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:08.213025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3227e-e182-4e3d-91e8-c74909761220 admin admin] PATH_INFO: `/v3/projects/defb3d3830774ddaa4e7e01012a817f6/users/0f3194718cf44dbeb218e29c0e69a3df/roles/ff293acb8551463aafe6970230042b89` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:08.214983 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3227e-e182-4e3d-91e8-c74909761220 admin admin] RBAC: Authorizing `identity:create_grant(project_id=defb3d3830774ddaa4e7e01012a817f6, user_id=0f3194718cf44dbeb218e29c0e69a3df, role_id=ff293acb8551463aafe6970230042b89)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:08.217622 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3227e-e182-4e3d-91e8-c74909761220 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:08.237647 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1472/2917] 10.222.0.58 () {60 vars in 1483 bytes} [Thu Sep 1 16:48:08 2022] PUT /identity/v3/projects/defb3d3830774ddaa4e7e01012a817f6/users/0f3194718cf44dbeb218e29c0e69a3df/roles/ff293acb8551463aafe6970230042b89 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:48:13.559670 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1dfffd-2509-4fd8-b07b-0700838d6e36 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:13.560254 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1dfffd-2509-4fd8-b07b-0700838d6e36 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:13.560254 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1dfffd-2509-4fd8-b07b-0700838d6e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:13.594820 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7a1dfffd-2509-4fd8-b07b-0700838d6e36 None None] MFA Rules not processed for user `0f3194718cf44dbeb218e29c0e69a3df`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:48:13.596163 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7a1dfffd-2509-4fd8-b07b-0700838d6e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:13.683743 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1446/2918] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:48:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:48:13.701454 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3227e-e182-4e3d-91e8-c74909761220 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:13.703778 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2aa3227e-e182-4e3d-91e8-c74909761220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:13.792013 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3227e-e182-4e3d-91e8-c74909761220 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:13.797694 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2725bf7-1e59-47fb-88b2-b1ecbd9e84b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:13.798616 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2725bf7-1e59-47fb-88b2-b1ecbd9e84b3 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:13.798896 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2725bf7-1e59-47fb-88b2-b1ecbd9e84b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:13.799163 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2725bf7-1e59-47fb-88b2-b1ecbd9e84b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:13.862668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2725bf7-1e59-47fb-88b2-b1ecbd9e84b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:13.864111 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2725bf7-1e59-47fb-88b2-b1ecbd9e84b3 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:13.872893 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1473/2919] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:48:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:48:22.025594 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1dfffd-2509-4fd8-b07b-0700838d6e36 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:22.027082 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7a1dfffd-2509-4fd8-b07b-0700838d6e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:22.077538 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:48:22.102841 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1dfffd-2509-4fd8-b07b-0700838d6e36 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:22.107640 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b60fb7-65b4-4839-86fc-c049d6f8e292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:22.108294 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b60fb7-65b4-4839-86fc-c049d6f8e292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:22.108579 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b60fb7-65b4-4839-86fc-c049d6f8e292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:22.108857 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b60fb7-65b4-4839-86fc-c049d6f8e292 admin admin] PATH_INFO: `/v3/users/e8f521436e924bf6a4571d3aeda481cf` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:22.110004 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b60fb7-65b4-4839-86fc-c049d6f8e292 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8f521436e924bf6a4571d3aeda481cf)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:22.111975 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b60fb7-65b4-4839-86fc-c049d6f8e292 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:22.171470 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d3b60fb7-65b4-4839-86fc-c049d6f8e292 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8f521436e924bf6a4571d3aeda481cf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:22.180669 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d3b60fb7-65b4-4839-86fc-c049d6f8e292 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8f521436e924bf6a4571d3aeda481cf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:22.181398 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d3b60fb7-65b4-4839-86fc-c049d6f8e292 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8f521436e924bf6a4571d3aeda481cf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:22.189030 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d3b60fb7-65b4-4839-86fc-c049d6f8e292 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8f521436e924bf6a4571d3aeda481cf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:22.204223 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d3b60fb7-65b4-4839-86fc-c049d6f8e292 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8f521436e924bf6a4571d3aeda481cf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:22.215087 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1447/2920] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:48:22 2022] DELETE /identity/v3/users/e8f521436e924bf6a4571d3aeda481cf => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:48:22.414398 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2725bf7-1e59-47fb-88b2-b1ecbd9e84b3 service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:22.427001 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2725bf7-1e59-47fb-88b2-b1ecbd9e84b3 service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:22.498109 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a2a27-b23b-44ec-adec-86b97d285666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:22.499022 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a2a27-b23b-44ec-adec-86b97d285666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:22.499420 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a2a27-b23b-44ec-adec-86b97d285666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:22.499815 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a2a27-b23b-44ec-adec-86b97d285666 admin admin] PATH_INFO: `/v3/projects/829ea07fd4c24f87b6eeb96e8a6952a0` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:22.501368 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a2a27-b23b-44ec-adec-86b97d285666 admin admin] RBAC: Authorizing `identity:delete_project(project_id=829ea07fd4c24f87b6eeb96e8a6952a0)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:22.504499 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a2a27-b23b-44ec-adec-86b97d285666 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:22.523881 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:48:22.568103 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-522a2a27-b23b-44ec-adec-86b97d285666 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '829ea07fd4c24f87b6eeb96e8a6952a0', 'request_id': 'req-522a2a27-b23b-44ec-adec-86b97d285666'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:22.575410 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-522a2a27-b23b-44ec-adec-86b97d285666 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '829ea07fd4c24f87b6eeb96e8a6952a0', 'request_id': 'req-522a2a27-b23b-44ec-adec-86b97d285666'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:22.580988 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-522a2a27-b23b-44ec-adec-86b97d285666 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '829ea07fd4c24f87b6eeb96e8a6952a0', 'request_id': 'req-522a2a27-b23b-44ec-adec-86b97d285666'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:22.595067 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-522a2a27-b23b-44ec-adec-86b97d285666 admin admin] The token cache is being invalidate because project 829ea07fd4c24f87b6eeb96e8a6952a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:48:22.595375 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-522a2a27-b23b-44ec-adec-86b97d285666 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:22.596137 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-522a2a27-b23b-44ec-adec-86b97d285666 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:48:22.600157 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1474/2921] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:48:22 2022] DELETE /identity/v3/projects/829ea07fd4c24f87b6eeb96e8a6952a0 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:48:32.207574 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:48:32.591434 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=19,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:48:39.845864 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95a902-8075-4980-8093-1e75b28c00bf None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:39.846553 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95a902-8075-4980-8093-1e75b28c00bf None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:39.846684 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95a902-8075-4980-8093-1e75b28c00bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:39.866606 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fd95a902-8075-4980-8093-1e75b28c00bf None None] MFA Rules not processed for user `8fa0f309a0b749b1a782d489f1f352af`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:48:39.867874 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fd95a902-8075-4980-8093-1e75b28c00bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:39.924847 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1448/2922] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:48:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:48:40.240068 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a2a27-b23b-44ec-adec-86b97d285666 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:40.243610 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-522a2a27-b23b-44ec-adec-86b97d285666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:40.314585 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a2a27-b23b-44ec-adec-86b97d285666 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:40.319320 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0796d-d2a7-4db7-b0e4-ea8e664a06aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:40.320213 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0796d-d2a7-4db7-b0e4-ea8e664a06aa service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:40.320497 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0796d-d2a7-4db7-b0e4-ea8e664a06aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:40.320951 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0796d-d2a7-4db7-b0e4-ea8e664a06aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:40.323210 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4ba0796d-d2a7-4db7-b0e4-ea8e664a06aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:40.337110 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0796d-d2a7-4db7-b0e4-ea8e664a06aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:40.339369 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0796d-d2a7-4db7-b0e4-ea8e664a06aa service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:40.352309 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1475/2923] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:48:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:48:40.926889 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd95a902-8075-4980-8093-1e75b28c00bf None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:40.928394 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fd95a902-8075-4980-8093-1e75b28c00bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:41.004887 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd95a902-8075-4980-8093-1e75b28c00bf None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:41.010376 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274586c5-c1b7-4891-93fb-2b93f6c5543b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12302cdb2854b0684179647a6dadf6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:41.010992 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274586c5-c1b7-4891-93fb-2b93f6c5543b service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:41.011271 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274586c5-c1b7-4891-93fb-2b93f6c5543b service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:41.011542 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274586c5-c1b7-4891-93fb-2b93f6c5543b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:41.013287 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-274586c5-c1b7-4891-93fb-2b93f6c5543b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:41.025980 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274586c5-c1b7-4891-93fb-2b93f6c5543b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:41.027597 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274586c5-c1b7-4891-93fb-2b93f6c5543b service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:41.037771 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1449/2924] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:48:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:48:43.619364 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0796d-d2a7-4db7-b0e4-ea8e664a06aa service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:43.621875 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4ba0796d-d2a7-4db7-b0e4-ea8e664a06aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:43.711230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0796d-d2a7-4db7-b0e4-ea8e664a06aa service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:43.716656 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490516f-9c4d-4d3e-98e4-7ccf8fad9284 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:43.717263 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490516f-9c4d-4d3e-98e4-7ccf8fad9284 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:43.717544 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490516f-9c4d-4d3e-98e4-7ccf8fad9284 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:43.717819 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490516f-9c4d-4d3e-98e4-7ccf8fad9284 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:43.719566 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b490516f-9c4d-4d3e-98e4-7ccf8fad9284 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:43.734077 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b490516f-9c4d-4d3e-98e4-7ccf8fad9284 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:43.735777 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b490516f-9c4d-4d3e-98e4-7ccf8fad9284 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:43.749121 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1476/2925] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:48:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:48:51.033387 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=57 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:48:53.742914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=60 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:48:54.603766 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274586c5-c1b7-4891-93fb-2b93f6c5543b service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:54.605298 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-274586c5-c1b7-4891-93fb-2b93f6c5543b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:54.681347 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274586c5-c1b7-4891-93fb-2b93f6c5543b service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:54.686952 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababb205-8d1b-42c5-bb11-882865e70458 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:54.687555 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababb205-8d1b-42c5-bb11-882865e70458 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:54.687838 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababb205-8d1b-42c5-bb11-882865e70458 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:54.688115 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababb205-8d1b-42c5-bb11-882865e70458 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:54.689861 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ababb205-8d1b-42c5-bb11-882865e70458 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:54.702678 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababb205-8d1b-42c5-bb11-882865e70458 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:54.704319 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababb205-8d1b-42c5-bb11-882865e70458 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:54.715643 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1450/2926] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:48:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:48:57.961155 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490516f-9c4d-4d3e-98e4-7ccf8fad9284 service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:57.981090 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490516f-9c4d-4d3e-98e4-7ccf8fad9284 service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:57.984780 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfeacdc-c2cf-4392-b1fe-d55df91c9dd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:57.985376 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfeacdc-c2cf-4392-b1fe-d55df91c9dd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:57.985654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfeacdc-c2cf-4392-b1fe-d55df91c9dd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:57.985926 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfeacdc-c2cf-4392-b1fe-d55df91c9dd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:57.987754 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ebfeacdc-c2cf-4392-b1fe-d55df91c9dd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:58.053525 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfeacdc-c2cf-4392-b1fe-d55df91c9dd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:58.055078 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfeacdc-c2cf-4392-b1fe-d55df91c9dd6 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:58.064614 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1477/2927] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:48:57 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:49:04.710685 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=28 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:49:08.060345 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=35 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:50:07.651916 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63e0e4-ae09-4403-9335-42a39827f030 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:50:07.652450 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63e0e4-ae09-4403-9335-42a39827f030 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:50:07.652564 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63e0e4-ae09-4403-9335-42a39827f030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:50:07.674550 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5c63e0e4-ae09-4403-9335-42a39827f030 None None] MFA Rules not processed for user `dd7b98536fa445cebd552fcbf54b757d`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:50:07.676054 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5c63e0e4-ae09-4403-9335-42a39827f030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:50:07.861951 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1451/2928] 10.222.0.58 () {58 vars in 967 bytes} [Thu Sep 1 16:50:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:50:07.903363 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfeacdc-c2cf-4392-b1fe-d55df91c9dd6 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:50:07.904813 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ebfeacdc-c2cf-4392-b1fe-d55df91c9dd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:50:07.976777 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfeacdc-c2cf-4392-b1fe-d55df91c9dd6 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:50:07.981758 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39530a7-e1ca-4c14-9f92-4e95183758d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e789695517c1437fb224f5a6195655f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:50:07.982392 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39530a7-e1ca-4c14-9f92-4e95183758d6 service swift] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:50:07.982675 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39530a7-e1ca-4c14-9f92-4e95183758d6 service swift] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:50:07.982945 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39530a7-e1ca-4c14-9f92-4e95183758d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:50:07.991645 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39530a7-e1ca-4c14-9f92-4e95183758d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:50:07.993242 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39530a7-e1ca-4c14-9f92-4e95183758d6 service swift] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:50:08.001413 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1478/2929] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Sep 1 16:50:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:50:14.336482 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c63e0e4-ae09-4403-9335-42a39827f030 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:50:14.373401 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c63e0e4-ae09-4403-9335-42a39827f030 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:50:14.379769 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb5cc0-23de-4892-a179-e3be79e52e70 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12302cdb2854b0684179647a6dadf6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:50:14.380956 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb5cc0-23de-4892-a179-e3be79e52e70 service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:50:14.381505 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb5cc0-23de-4892-a179-e3be79e52e70 service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:50:14.382140 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb5cc0-23de-4892-a179-e3be79e52e70 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:50:14.385018 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1fdb5cc0-23de-4892-a179-e3be79e52e70 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:50:14.421310 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39530a7-e1ca-4c14-9f92-4e95183758d6 service swift] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:50:14.423761 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d39530a7-e1ca-4c14-9f92-4e95183758d6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:50:14.425863 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdb5cc0-23de-4892-a179-e3be79e52e70 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:50:14.429789 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdb5cc0-23de-4892-a179-e3be79e52e70 service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:50:14.456731 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1452/2930] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:50:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:50:14.560183 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39530a7-e1ca-4c14-9f92-4e95183758d6 service swift] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:50:14.565103 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15efd53-fd91-4eeb-83d8-9ac092a81b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:50:14.565722 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15efd53-fd91-4eeb-83d8-9ac092a81b02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:50:14.566003 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15efd53-fd91-4eeb-83d8-9ac092a81b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:50:14.566429 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15efd53-fd91-4eeb-83d8-9ac092a81b02 admin admin] PATH_INFO: `/v3/users/fac40ee8cdc9495aa1c94f483f6fd6c5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:50:14.567704 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15efd53-fd91-4eeb-83d8-9ac092a81b02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fac40ee8cdc9495aa1c94f483f6fd6c5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:50:14.569670 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15efd53-fd91-4eeb-83d8-9ac092a81b02 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:50:14.622953 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c15efd53-fd91-4eeb-83d8-9ac092a81b02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fac40ee8cdc9495aa1c94f483f6fd6c5'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:14.630141 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c15efd53-fd91-4eeb-83d8-9ac092a81b02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fac40ee8cdc9495aa1c94f483f6fd6c5'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:14.630889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c15efd53-fd91-4eeb-83d8-9ac092a81b02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fac40ee8cdc9495aa1c94f483f6fd6c5'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:14.631532 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c15efd53-fd91-4eeb-83d8-9ac092a81b02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fac40ee8cdc9495aa1c94f483f6fd6c5'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:14.636982 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:50:14.648739 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c15efd53-fd91-4eeb-83d8-9ac092a81b02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fac40ee8cdc9495aa1c94f483f6fd6c5'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:14.668534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1479/2931] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:50:14 2022] DELETE /identity/v3/users/fac40ee8cdc9495aa1c94f483f6fd6c5 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:50:14.863957 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb5cc0-23de-4892-a179-e3be79e52e70 service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:50:14.878772 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb5cc0-23de-4892-a179-e3be79e52e70 service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:50:14.938739 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82086f-74dd-498f-a5a2-eb4bd81579cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:50:14.939377 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82086f-74dd-498f-a5a2-eb4bd81579cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:50:14.939658 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82086f-74dd-498f-a5a2-eb4bd81579cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:50:14.939929 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82086f-74dd-498f-a5a2-eb4bd81579cc admin admin] PATH_INFO: `/v3/projects/1f62976899a74162b21efe5580758fd9` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:50:14.941058 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c82086f-74dd-498f-a5a2-eb4bd81579cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f62976899a74162b21efe5580758fd9)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:50:14.942985 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c82086f-74dd-498f-a5a2-eb4bd81579cc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:50:14.991288 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5c82086f-74dd-498f-a5a2-eb4bd81579cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f62976899a74162b21efe5580758fd9', 'request_id': 'req-5c82086f-74dd-498f-a5a2-eb4bd81579cc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:14.997496 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5c82086f-74dd-498f-a5a2-eb4bd81579cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f62976899a74162b21efe5580758fd9', 'request_id': 'req-5c82086f-74dd-498f-a5a2-eb4bd81579cc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:15.005841 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5c82086f-74dd-498f-a5a2-eb4bd81579cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f62976899a74162b21efe5580758fd9', 'request_id': 'req-5c82086f-74dd-498f-a5a2-eb4bd81579cc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:15.017089 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5c82086f-74dd-498f-a5a2-eb4bd81579cc admin admin] The token cache is being invalidate because project 1f62976899a74162b21efe5580758fd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:50:15.017366 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5c82086f-74dd-498f-a5a2-eb4bd81579cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:15.018024 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5c82086f-74dd-498f-a5a2-eb4bd81579cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:15.019543 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1453/2932] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:50:14 2022] DELETE /identity/v3/projects/1f62976899a74162b21efe5580758fd9 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:50:15.024383 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15efd53-fd91-4eeb-83d8-9ac092a81b02 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:50:15.025809 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c15efd53-fd91-4eeb-83d8-9ac092a81b02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:50:15.039168 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15efd53-fd91-4eeb-83d8-9ac092a81b02 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:50:15.116748 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81266a-5d5b-4cb0-ae1d-489d3752ad6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:50:15.117282 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81266a-5d5b-4cb0-ae1d-489d3752ad6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:50:15.117566 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81266a-5d5b-4cb0-ae1d-489d3752ad6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:50:15.117832 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81266a-5d5b-4cb0-ae1d-489d3752ad6f admin admin] PATH_INFO: `/v3/users/a69e65a5607f4d14b3980e9ebaab2e08` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:50:15.119012 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81266a-5d5b-4cb0-ae1d-489d3752ad6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a69e65a5607f4d14b3980e9ebaab2e08)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:50:15.120914 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81266a-5d5b-4cb0-ae1d-489d3752ad6f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:50:15.169610 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5d81266a-5d5b-4cb0-ae1d-489d3752ad6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a69e65a5607f4d14b3980e9ebaab2e08'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:15.176654 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5d81266a-5d5b-4cb0-ae1d-489d3752ad6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a69e65a5607f4d14b3980e9ebaab2e08'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:15.177320 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5d81266a-5d5b-4cb0-ae1d-489d3752ad6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a69e65a5607f4d14b3980e9ebaab2e08'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:15.177918 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5d81266a-5d5b-4cb0-ae1d-489d3752ad6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a69e65a5607f4d14b3980e9ebaab2e08'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:15.190850 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5d81266a-5d5b-4cb0-ae1d-489d3752ad6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a69e65a5607f4d14b3980e9ebaab2e08'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:15.206493 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1480/2933] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:50:15 2022] DELETE /identity/v3/users/a69e65a5607f4d14b3980e9ebaab2e08 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:50:15.380423 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82086f-74dd-498f-a5a2-eb4bd81579cc admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:50:15.382699 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5c82086f-74dd-498f-a5a2-eb4bd81579cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:50:15.390640 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:50:15.397248 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82086f-74dd-498f-a5a2-eb4bd81579cc admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:50:15.458370 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25795d3-1b8e-4f90-9750-067375ef59a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:50:15.459008 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25795d3-1b8e-4f90-9750-067375ef59a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:50:15.459291 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25795d3-1b8e-4f90-9750-067375ef59a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:50:15.459615 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25795d3-1b8e-4f90-9750-067375ef59a9 admin admin] PATH_INFO: `/v3/projects/81558e2e92b24d02b1040a545d867a36` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:50:15.460892 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25795d3-1b8e-4f90-9750-067375ef59a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81558e2e92b24d02b1040a545d867a36)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:50:15.462845 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25795d3-1b8e-4f90-9750-067375ef59a9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:50:15.512716 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f25795d3-1b8e-4f90-9750-067375ef59a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81558e2e92b24d02b1040a545d867a36', 'request_id': 'req-f25795d3-1b8e-4f90-9750-067375ef59a9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:15.519496 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f25795d3-1b8e-4f90-9750-067375ef59a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81558e2e92b24d02b1040a545d867a36', 'request_id': 'req-f25795d3-1b8e-4f90-9750-067375ef59a9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:15.529523 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f25795d3-1b8e-4f90-9750-067375ef59a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81558e2e92b24d02b1040a545d867a36', 'request_id': 'req-f25795d3-1b8e-4f90-9750-067375ef59a9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:15.540754 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f25795d3-1b8e-4f90-9750-067375ef59a9 admin admin] The token cache is being invalidate because project 81558e2e92b24d02b1040a545d867a36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:50:15.541043 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f25795d3-1b8e-4f90-9750-067375ef59a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:15.541702 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f25795d3-1b8e-4f90-9750-067375ef59a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:15.543334 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1454/2934] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:50:15 2022] DELETE /identity/v3/projects/81558e2e92b24d02b1040a545d867a36 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:50:25.202913 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=59,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:50:25.537575 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=50,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:51:08.118256 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81266a-5d5b-4cb0-ae1d-489d3752ad6f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:51:08.119721 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5d81266a-5d5b-4cb0-ae1d-489d3752ad6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:51:08.193713 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81266a-5d5b-4cb0-ae1d-489d3752ad6f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:51:08.257593 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bfb885-4db5-4837-a77d-3c4525e7fe35 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12302cdb2854b0684179647a6dadf6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:51:08.258238 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfb885-4db5-4837-a77d-3c4525e7fe35 service placement] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:51:08.258527 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfb885-4db5-4837-a77d-3c4525e7fe35 service placement] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:51:08.258811 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfb885-4db5-4837-a77d-3c4525e7fe35 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:51:08.260626 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-74bfb885-4db5-4837-a77d-3c4525e7fe35 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:51:08.273631 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bfb885-4db5-4837-a77d-3c4525e7fe35 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:51:08.275280 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bfb885-4db5-4837-a77d-3c4525e7fe35 service placement] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:51:08.286025 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1481/2935] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:51:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:51:18.194740 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25795d3-1b8e-4f90-9750-067375ef59a9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:51:18.196246 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f25795d3-1b8e-4f90-9750-067375ef59a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:51:18.279234 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25795d3-1b8e-4f90-9750-067375ef59a9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:51:18.281272 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=58 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:51:18.283555 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6302e7-a855-4154-b9b5-f9343f28d4e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:51:18.284177 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6302e7-a855-4154-b9b5-f9343f28d4e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:51:18.284455 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6302e7-a855-4154-b9b5-f9343f28d4e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:51:18.284728 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6302e7-a855-4154-b9b5-f9343f28d4e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:51:18.286549 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0a6302e7-a855-4154-b9b5-f9343f28d4e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:51:18.376902 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6302e7-a855-4154-b9b5-f9343f28d4e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:51:18.378518 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6302e7-a855-4154-b9b5-f9343f28d4e9 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:51:18.401234 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1455/2936] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:51:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:51:19.842115 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bfb885-4db5-4837-a77d-3c4525e7fe35 service placement] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:51:19.852197 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bfb885-4db5-4837-a77d-3c4525e7fe35 service placement] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:51:19.857076 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c296225-9a98-4b80-9bde-f8a95022574d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:51:19.857747 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c296225-9a98-4b80-9bde-f8a95022574d service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:51:19.858056 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c296225-9a98-4b80-9bde-f8a95022574d service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:51:19.858534 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c296225-9a98-4b80-9bde-f8a95022574d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:51:19.860486 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4c296225-9a98-4b80-9bde-f8a95022574d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:51:19.875903 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c296225-9a98-4b80-9bde-f8a95022574d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:51:19.877517 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c296225-9a98-4b80-9bde-f8a95022574d service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:51:19.890303 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1482/2937] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:51:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:51:22.213767 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f239c-6364-471b-bd65-e831a4111ca5 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:51:22.214308 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f239c-6364-471b-bd65-e831a4111ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:51:22.214419 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f239c-6364-471b-bd65-e831a4111ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:51:22.234139 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e60f239c-6364-471b-bd65-e831a4111ca5 None None] MFA Rules not processed for user `8fa0f309a0b749b1a782d489f1f352af`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:51:22.235368 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e60f239c-6364-471b-bd65-e831a4111ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:51:22.283775 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:51:22.291961 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1456/2938] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:51:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:51:22.620047 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c296225-9a98-4b80-9bde-f8a95022574d service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:51:22.629536 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c296225-9a98-4b80-9bde-f8a95022574d service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:51:22.633737 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d619c-bce9-4e7d-831c-089bf52dce48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:51:22.634376 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d619c-bce9-4e7d-831c-089bf52dce48 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:51:22.634672 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d619c-bce9-4e7d-831c-089bf52dce48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:51:22.634938 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d619c-bce9-4e7d-831c-089bf52dce48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:51:22.636809 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6c1d619c-bce9-4e7d-831c-089bf52dce48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:51:22.649839 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d619c-bce9-4e7d-831c-089bf52dce48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:51:22.651321 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d619c-bce9-4e7d-831c-089bf52dce48 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:51:22.660465 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1483/2939] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:51:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:51:25.252099 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f239c-6364-471b-bd65-e831a4111ca5 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:51:25.265635 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f239c-6364-471b-bd65-e831a4111ca5 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:51:25.271897 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438cee3d-7e36-44e5-b212-074468d3b480 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:51:25.272954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cee3d-7e36-44e5-b212-074468d3b480 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:51:25.273449 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cee3d-7e36-44e5-b212-074468d3b480 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:51:25.273927 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cee3d-7e36-44e5-b212-074468d3b480 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:51:25.276825 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-438cee3d-7e36-44e5-b212-074468d3b480 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:51:25.292550 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438cee3d-7e36-44e5-b212-074468d3b480 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:51:25.294011 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438cee3d-7e36-44e5-b212-074468d3b480 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:51:25.303559 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1457/2940] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:51:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:51:32.656228 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=12 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:51:35.298922 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:53:31.392606 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d619c-bce9-4e7d-831c-089bf52dce48 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:53:31.394002 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6c1d619c-bce9-4e7d-831c-089bf52dce48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:53:31.476833 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d619c-bce9-4e7d-831c-089bf52dce48 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:53:31.480423 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554cec0-0358-4d98-8884-6e96522a25ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d4bd0a456341f78b5e476c97a9165c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:53:31.481021 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554cec0-0358-4d98-8884-6e96522a25ae service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:53:31.481307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554cec0-0358-4d98-8884-6e96522a25ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:53:31.481666 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554cec0-0358-4d98-8884-6e96522a25ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:53:31.483362 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0554cec0-0358-4d98-8884-6e96522a25ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:53:31.495116 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554cec0-0358-4d98-8884-6e96522a25ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:53:31.496670 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554cec0-0358-4d98-8884-6e96522a25ae service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:53:31.507068 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1484/2941] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:53:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:53:41.502504 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=35 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:53:42.582466 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb044a-b931-4a11-bea1-42790a241f11 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:53:42.583055 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb044a-b931-4a11-bea1-42790a241f11 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:53:42.583055 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb044a-b931-4a11-bea1-42790a241f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:53:42.601925 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-adcb044a-b931-4a11-bea1-42790a241f11 None None] MFA Rules not processed for user `8fa0f309a0b749b1a782d489f1f352af`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:53:42.603281 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-adcb044a-b931-4a11-bea1-42790a241f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:53:42.610097 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1458/2942] 10.222.0.58 () {58 vars in 958 bytes} [Thu Sep 1 16:53:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:53:43.449423 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554cec0-0358-4d98-8884-6e96522a25ae service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:53:43.450991 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0554cec0-0358-4d98-8884-6e96522a25ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:53:43.467161 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554cec0-0358-4d98-8884-6e96522a25ae service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:53:43.473399 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac99f2-db4e-48b5-8bbe-4f25944e602f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12302cdb2854b0684179647a6dadf6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:53:43.474083 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac99f2-db4e-48b5-8bbe-4f25944e602f service placement] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:53:43.474447 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac99f2-db4e-48b5-8bbe-4f25944e602f service placement] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:53:43.474673 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac99f2-db4e-48b5-8bbe-4f25944e602f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:53:43.476446 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-eeac99f2-db4e-48b5-8bbe-4f25944e602f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:53:43.491676 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac99f2-db4e-48b5-8bbe-4f25944e602f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:53:43.493278 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac99f2-db4e-48b5-8bbe-4f25944e602f service placement] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:53:43.505395 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1485/2943] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:53:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:53:46.221391 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb044a-b931-4a11-bea1-42790a241f11 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:53:46.223877 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-adcb044a-b931-4a11-bea1-42790a241f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:53:46.243126 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb044a-b931-4a11-bea1-42790a241f11 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:53:46.248814 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0455c34-ba2c-4fb0-bb57-ff846172fce5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:53:46.249435 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0455c34-ba2c-4fb0-bb57-ff846172fce5 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:53:46.249714 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0455c34-ba2c-4fb0-bb57-ff846172fce5 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:53:46.249984 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0455c34-ba2c-4fb0-bb57-ff846172fce5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:53:46.252452 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a0455c34-ba2c-4fb0-bb57-ff846172fce5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:53:46.265625 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0455c34-ba2c-4fb0-bb57-ff846172fce5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:53:46.268266 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0455c34-ba2c-4fb0-bb57-ff846172fce5 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:53:46.283705 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1459/2944] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:53:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:53:53.500608 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=6 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:53:56.276343 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=9 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:53:58.048843 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac99f2-db4e-48b5-8bbe-4f25944e602f service placement] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:53:58.051285 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-eeac99f2-db4e-48b5-8bbe-4f25944e602f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:53:58.070445 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac99f2-db4e-48b5-8bbe-4f25944e602f service placement] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:53:58.076219 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9d38c-5a41-4911-9978-17aa7a846413 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:53:58.076833 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9d38c-5a41-4911-9978-17aa7a846413 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:53:58.077109 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9d38c-5a41-4911-9978-17aa7a846413 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:53:58.077381 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9d38c-5a41-4911-9978-17aa7a846413 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:53:58.079191 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5ec9d38c-5a41-4911-9978-17aa7a846413 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:53:58.093663 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec9d38c-5a41-4911-9978-17aa7a846413 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:53:58.095357 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec9d38c-5a41-4911-9978-17aa7a846413 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:53:58.111578 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1486/2945] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:53:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:54:01.590894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0455c34-ba2c-4fb0-bb57-ff846172fce5 service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:54:01.600918 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0455c34-ba2c-4fb0-bb57-ff846172fce5 service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:54:01.604759 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ae264-8050-41be-b984-89f5751da79b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:54:01.605353 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ae264-8050-41be-b984-89f5751da79b service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:54:01.605627 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ae264-8050-41be-b984-89f5751da79b service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:54:01.605894 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ae264-8050-41be-b984-89f5751da79b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:54:01.607639 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9c5ae264-8050-41be-b984-89f5751da79b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:54:01.677608 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5ae264-8050-41be-b984-89f5751da79b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:54:01.679108 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5ae264-8050-41be-b984-89f5751da79b service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:54:01.688734 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1460/2946] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:54:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:54:08.106330 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=6 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:54:11.684424 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=30 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:55:56.137022 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9d38c-5a41-4911-9978-17aa7a846413 service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:55:56.149254 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9d38c-5a41-4911-9978-17aa7a846413 service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:55:56.154930 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50286c66-62da-4a29-9557-6e1af7cd16f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f0c99fdf8bd4331ad7e641ee38554c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:55:56.155604 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50286c66-62da-4a29-9557-6e1af7cd16f5 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:55:56.155889 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50286c66-62da-4a29-9557-6e1af7cd16f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:55:56.156164 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50286c66-62da-4a29-9557-6e1af7cd16f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:55:56.157964 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-50286c66-62da-4a29-9557-6e1af7cd16f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:55:56.171050 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50286c66-62da-4a29-9557-6e1af7cd16f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:55:56.172668 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50286c66-62da-4a29-9557-6e1af7cd16f5 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:55:56.184000 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1487/2947] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:55:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:56:06.179295 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=6 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:56:06.896365 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ae264-8050-41be-b984-89f5751da79b service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:56:06.897931 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9c5ae264-8050-41be-b984-89f5751da79b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:56:06.909148 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ae264-8050-41be-b984-89f5751da79b service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:56:06.912788 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973adc44-6502-496c-b0ec-09d9b957a5c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '943fae29c5dc41f2911bbe051c74a361', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:56:06.913395 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973adc44-6502-496c-b0ec-09d9b957a5c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:56:06.913678 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973adc44-6502-496c-b0ec-09d9b957a5c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:56:06.913954 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973adc44-6502-496c-b0ec-09d9b957a5c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:56:06.915783 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-973adc44-6502-496c-b0ec-09d9b957a5c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:56:06.987517 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973adc44-6502-496c-b0ec-09d9b957a5c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:56:06.989254 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973adc44-6502-496c-b0ec-09d9b957a5c9 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:56:06.999144 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1461/2948] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:56:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:56:10.049788 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50286c66-62da-4a29-9557-6e1af7cd16f5 service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:56:10.062307 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50286c66-62da-4a29-9557-6e1af7cd16f5 service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:56:10.068772 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494aa23-c2b6-4dd9-a0a3-d05382ad1057 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12302cdb2854b0684179647a6dadf6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:56:10.069406 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494aa23-c2b6-4dd9-a0a3-d05382ad1057 service placement] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:56:10.069688 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494aa23-c2b6-4dd9-a0a3-d05382ad1057 service placement] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:56:10.069966 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494aa23-c2b6-4dd9-a0a3-d05382ad1057 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:56:10.071798 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b494aa23-c2b6-4dd9-a0a3-d05382ad1057 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:56:10.085175 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494aa23-c2b6-4dd9-a0a3-d05382ad1057 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:56:10.086825 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494aa23-c2b6-4dd9-a0a3-d05382ad1057 service placement] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:56:10.099538 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1488/2949] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:56:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:56:12.025130 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973adc44-6502-496c-b0ec-09d9b957a5c9 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:56:12.041811 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973adc44-6502-496c-b0ec-09d9b957a5c9 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:56:12.051344 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40270f66-d2cd-4edb-a74f-b7dc0a003803 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12302cdb2854b0684179647a6dadf6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:56:12.051989 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40270f66-d2cd-4edb-a74f-b7dc0a003803 service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:56:12.052269 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40270f66-d2cd-4edb-a74f-b7dc0a003803 service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:56:12.052726 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40270f66-d2cd-4edb-a74f-b7dc0a003803 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:56:12.062177 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40270f66-d2cd-4edb-a74f-b7dc0a003803 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:56:12.064225 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40270f66-d2cd-4edb-a74f-b7dc0a003803 service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:56:12.078505 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1462/2950] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:56:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:56:13.596120 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494aa23-c2b6-4dd9-a0a3-d05382ad1057 service placement] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:56:13.610973 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494aa23-c2b6-4dd9-a0a3-d05382ad1057 service placement] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:56:13.618315 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd0643f-5a0e-47f1-b45d-915bfbd947c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12302cdb2854b0684179647a6dadf6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:56:13.618934 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0643f-5a0e-47f1-b45d-915bfbd947c2 service placement] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:56:13.619355 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0643f-5a0e-47f1-b45d-915bfbd947c2 service placement] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:56:13.619635 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0643f-5a0e-47f1-b45d-915bfbd947c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:56:13.621365 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2dd0643f-5a0e-47f1-b45d-915bfbd947c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:56:13.634699 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd0643f-5a0e-47f1-b45d-915bfbd947c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:56:13.636309 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd0643f-5a0e-47f1-b45d-915bfbd947c2 service placement] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:56:13.647786 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1489/2951] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 16:56:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:56:13.682430 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40270f66-d2cd-4edb-a74f-b7dc0a003803 service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:56:13.683925 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-40270f66-d2cd-4edb-a74f-b7dc0a003803 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:56:13.700615 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40270f66-d2cd-4edb-a74f-b7dc0a003803 service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:56:13.707378 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade0396-2592-4aa7-a3da-b1acaa8ff1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:56:13.708018 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade0396-2592-4aa7-a3da-b1acaa8ff1a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:56:13.708301 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade0396-2592-4aa7-a3da-b1acaa8ff1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:56:13.708576 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade0396-2592-4aa7-a3da-b1acaa8ff1a7 admin admin] PATH_INFO: `/v3/users/8fa0f309a0b749b1a782d489f1f352af` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:56:13.709699 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade0396-2592-4aa7-a3da-b1acaa8ff1a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fa0f309a0b749b1a782d489f1f352af)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:56:13.711681 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade0396-2592-4aa7-a3da-b1acaa8ff1a7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:56:13.760828 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cade0396-2592-4aa7-a3da-b1acaa8ff1a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fa0f309a0b749b1a782d489f1f352af'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:13.768320 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cade0396-2592-4aa7-a3da-b1acaa8ff1a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fa0f309a0b749b1a782d489f1f352af'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:13.769026 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cade0396-2592-4aa7-a3da-b1acaa8ff1a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fa0f309a0b749b1a782d489f1f352af'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:13.776208 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cade0396-2592-4aa7-a3da-b1acaa8ff1a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fa0f309a0b749b1a782d489f1f352af'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:13.789105 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cade0396-2592-4aa7-a3da-b1acaa8ff1a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fa0f309a0b749b1a782d489f1f352af'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:13.796262 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1463/2952] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:56:13 2022] DELETE /identity/v3/users/8fa0f309a0b749b1a782d489f1f352af => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:56:13.985890 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd0643f-5a0e-47f1-b45d-915bfbd947c2 service placement] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:56:13.998020 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd0643f-5a0e-47f1-b45d-915bfbd947c2 service placement] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:56:14.058793 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd76994-3d22-4991-8bc9-83e8763ab633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:56:14.059436 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd76994-3d22-4991-8bc9-83e8763ab633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:56:14.059719 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd76994-3d22-4991-8bc9-83e8763ab633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:56:14.059994 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd76994-3d22-4991-8bc9-83e8763ab633 admin admin] PATH_INFO: `/v3/projects/20a271a8da6d423db38d5730006af839` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:56:14.061224 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd76994-3d22-4991-8bc9-83e8763ab633 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20a271a8da6d423db38d5730006af839)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:56:14.063230 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd76994-3d22-4991-8bc9-83e8763ab633 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:56:14.114182 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-afd76994-3d22-4991-8bc9-83e8763ab633 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20a271a8da6d423db38d5730006af839', 'request_id': 'req-afd76994-3d22-4991-8bc9-83e8763ab633'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:14.120715 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-afd76994-3d22-4991-8bc9-83e8763ab633 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20a271a8da6d423db38d5730006af839', 'request_id': 'req-afd76994-3d22-4991-8bc9-83e8763ab633'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:14.126338 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-afd76994-3d22-4991-8bc9-83e8763ab633 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20a271a8da6d423db38d5730006af839', 'request_id': 'req-afd76994-3d22-4991-8bc9-83e8763ab633'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:14.138876 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-afd76994-3d22-4991-8bc9-83e8763ab633 admin admin] The token cache is being invalidate because project 20a271a8da6d423db38d5730006af839 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:56:14.139169 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-afd76994-3d22-4991-8bc9-83e8763ab633 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:14.139811 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-afd76994-3d22-4991-8bc9-83e8763ab633 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:14.141262 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1490/2953] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:56:13 2022] DELETE /identity/v3/projects/20a271a8da6d423db38d5730006af839 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:56:14.145421 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade0396-2592-4aa7-a3da-b1acaa8ff1a7 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:56:14.146839 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cade0396-2592-4aa7-a3da-b1acaa8ff1a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:56:14.161665 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade0396-2592-4aa7-a3da-b1acaa8ff1a7 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:56:14.213971 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:56:14.225059 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c056fc-7852-43f7-8efc-1e7596243df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:56:14.225694 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c056fc-7852-43f7-8efc-1e7596243df1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:56:14.225971 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c056fc-7852-43f7-8efc-1e7596243df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:56:14.226264 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c056fc-7852-43f7-8efc-1e7596243df1 admin admin] PATH_INFO: `/v3/users/0f3194718cf44dbeb218e29c0e69a3df` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:56:14.227432 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c056fc-7852-43f7-8efc-1e7596243df1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f3194718cf44dbeb218e29c0e69a3df)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:56:14.229359 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c056fc-7852-43f7-8efc-1e7596243df1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:56:14.280731 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e8c056fc-7852-43f7-8efc-1e7596243df1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f3194718cf44dbeb218e29c0e69a3df'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:14.287757 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e8c056fc-7852-43f7-8efc-1e7596243df1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f3194718cf44dbeb218e29c0e69a3df'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:14.288457 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e8c056fc-7852-43f7-8efc-1e7596243df1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f3194718cf44dbeb218e29c0e69a3df'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:14.297068 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e8c056fc-7852-43f7-8efc-1e7596243df1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f3194718cf44dbeb218e29c0e69a3df'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:14.309880 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e8c056fc-7852-43f7-8efc-1e7596243df1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f3194718cf44dbeb218e29c0e69a3df'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:14.317033 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1464/2954] 10.222.0.58 () {58 vars in 1224 bytes} [Thu Sep 1 16:56:14 2022] DELETE /identity/v3/users/0f3194718cf44dbeb218e29c0e69a3df => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:56:14.495652 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd76994-3d22-4991-8bc9-83e8763ab633 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:56:14.497136 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-afd76994-3d22-4991-8bc9-83e8763ab633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:56:14.510847 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd76994-3d22-4991-8bc9-83e8763ab633 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:56:14.575984 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f8993a-0156-4751-a301-9427c72b2ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe3a5506b35415a9373071c0b976e34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2c38221adde4053b2b37201b4051903', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:56:14.577164 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8993a-0156-4751-a301-9427c72b2ddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:56:14.577394 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8993a-0156-4751-a301-9427c72b2ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:56:14.577670 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8993a-0156-4751-a301-9427c72b2ddb admin admin] PATH_INFO: `/v3/projects/defb3d3830774ddaa4e7e01012a817f6` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:56:14.578828 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f8993a-0156-4751-a301-9427c72b2ddb admin admin] RBAC: Authorizing `identity:delete_project(project_id=defb3d3830774ddaa4e7e01012a817f6)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:56:14.581411 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f8993a-0156-4751-a301-9427c72b2ddb admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:56:14.588508 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:56:14.632953 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-68f8993a-0156-4751-a301-9427c72b2ddb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'defb3d3830774ddaa4e7e01012a817f6', 'request_id': 'req-68f8993a-0156-4751-a301-9427c72b2ddb'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:14.639831 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-68f8993a-0156-4751-a301-9427c72b2ddb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'defb3d3830774ddaa4e7e01012a817f6', 'request_id': 'req-68f8993a-0156-4751-a301-9427c72b2ddb'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:14.646068 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-68f8993a-0156-4751-a301-9427c72b2ddb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'defb3d3830774ddaa4e7e01012a817f6', 'request_id': 'req-68f8993a-0156-4751-a301-9427c72b2ddb'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:14.661804 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-68f8993a-0156-4751-a301-9427c72b2ddb admin admin] The token cache is being invalidate because project defb3d3830774ddaa4e7e01012a817f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:56:14.662115 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-68f8993a-0156-4751-a301-9427c72b2ddb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:14.662821 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-68f8993a-0156-4751-a301-9427c72b2ddb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:14.664328 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1491/2955] 10.222.0.58 () {58 vars in 1233 bytes} [Thu Sep 1 16:56:14 2022] DELETE /identity/v3/projects/defb3d3830774ddaa4e7e01012a817f6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:56:24.312607 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:56:24.657927 dv-cinder-852208-16-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:01:20.075134 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c056fc-7852-43f7-8efc-1e7596243df1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:01:20.076590 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e8c056fc-7852-43f7-8efc-1e7596243df1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:01:20.217794 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c056fc-7852-43f7-8efc-1e7596243df1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:01:20.275611 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaace86-e931-45aa-bc00-1c48ed645aec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12302cdb2854b0684179647a6dadf6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '479530ced580426b85d1fc07af43417f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:01:20.276220 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaace86-e931-45aa-bc00-1c48ed645aec service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:01:20.276502 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaace86-e931-45aa-bc00-1c48ed645aec service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:01:20.276776 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaace86-e931-45aa-bc00-1c48ed645aec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:01:20.278550 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6aaace86-e931-45aa-bc00-1c48ed645aec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:01:20.289436 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaace86-e931-45aa-bc00-1c48ed645aec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:01:20.291020 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaace86-e931-45aa-bc00-1c48ed645aec service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:01:20.301137 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1465/2956] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Sep 1 17:01:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:01:30.296484 dv-cinder-852208-16-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}